Cannot install fairseq==0.10.2 with python 3.9

I don't seem to be able to install fairseq==0.10.2 with python 3.9 on macOS 11.3
$ pip install fairseq==0.10.2
Collecting fairseq==0.10.2
  Using cached fairseq-0.10.2.tar.gz (938 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/louismartin/miniconda3/envs/dev3.9/bin/python /Users/louismartin/miniconda3/envs/dev3.9/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /var/folders/_4/yxn93yzn14g0_0k2rmctcc1x8lvpn4/T/tmpok_9kdi5
       cwd: /private/var/folders/_4/yxn93yzn14g0_0k2rmctcc1x8lvpn4/T/pip-install-c4qbk_3i/fairseq_751efca8a5d547ada5c29fe71ec11ded
  Complete output (31 lines):
  Traceback (most recent call last):
    File "setup.py", line 214, in <module>
      do_setup(package_data)
    File "setup.py", line 136, in do_setup
      setup(
    File "/private/var/folders/_4/yxn93yzn14g0_0k2rmctcc1x8lvpn4/T/pip-build-env-gi7j5w4c/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 152, in setup
      _install_setup_requires(attrs)
    File "/private/var/folders/_4/yxn93yzn14g0_0k2rmctcc1x8lvpn4/T/pip-build-env-gi7j5w4c/overlay/lib/python3.9/site-packages/setuptools/__init__.py", line 147, in _install_setup_requires
      dist.fetch_build_eggs(dist.setup_requires)
    File "/private/var/folders/_4/yxn93yzn14g0_0k2rmctcc1x8lvpn4/T/pip-build-env-gi7j5w4c/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 60, in fetch_build_eggs
      raise SetupRequirementsError(specifier_list)
  setuptools.build_meta.SetupRequirementsError: ['cython', 'numpy', 'setuptools>=18.0']

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/Users/louismartin/miniconda3/envs/dev3.9/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
      main()
    File "/Users/louismartin/miniconda3/envs/dev3.9/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/Users/louismartin/miniconda3/envs/dev3.9/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py", line 114, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/private/var/folders/_4/yxn93yzn14g0_0k2rmctcc1x8lvpn4/T/pip-build-env-gi7j5w4c/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 154, in get_requires_for_build_wheel
      return self._get_build_requires(
    File "/private/var/folders/_4/yxn93yzn14g0_0k2rmctcc1x8lvpn4/T/pip-build-env-gi7j5w4c/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 135, in _get_build_requires
      self.run_setup()
    File "/private/var/folders/_4/yxn93yzn14g0_0k2rmctcc1x8lvpn4/T/pip-build-env-gi7j5w4c/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 150, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 217, in <module>
      os.unlink(fairseq_examples)
  PermissionError: [Errno 1] Operation not permitted: 'fairseq/examples'
  ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/02/59/1f466d82d64482e2809a167e0067979c23057b3752e668537d5ff24453c9/fairseq-0.10.2.tar.gz#sha256=45b90d8ccc3f5a4623a523fe2d0465f54413d03fd1ec9a9d7af0461148ca1a68 (from https://pypi.org/simple/fairseq/). Command errored out with exit status 1: /Users/louismartin/miniconda3/envs/dev3.9/bin/python /Users/louismartin/miniconda3/envs/dev3.9/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /var/folders/_4/yxn93yzn14g0_0k2rmctcc1x8lvpn4/T/tmpok_9kdi5 Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement fairseq==0.10.2
ERROR: No matching distribution found for fairseq==0.10.2

Seems to be due to this line

I reproduced the issue on ubuntu 20.04 with python 3.9
The error does not occur with ubuntu 20.04 and python 3.8 so it is probably due to python 3.9.

Hi @myleott any news on when a new pip version will be released fixing compatibility for python 3.9?
I'm asking because I get issues on my repo about python 3.9 compatibility.

It's not just a problem with python 3.9. It appears even for 3.8, 3.7, 3.6.

Facing the same issue with python 3.6 . Any updates on this?

Facing the same issue with python 3.6 . Any updates on this? @leeway00 @louismartin

저는 도커 alpine에서 돌리다 문제가 발생했었는데, torch 이미지로 도커파일 바꿔서 해결했어요.. 같은 상황이실지는 모르겠네요

Facing the same issue with python 3.6 . Any updates on this? @leeway00 @louismartin

저는 도커 alpine에서 돌리다 문제가 발생했었는데, torch 이미지로 도커파일 바꿔서 해결했어요.. 같은 상황이실지는 모르겠네요

Ah! I'm unfortunately in a different situation. I'm on conda env. Anyways, thank you for your help. :)

Having the same issue. Would love to hear if you have any solution for this?

Hi, I have the same issue and would love to hear any solutions! I am using Python 3.8 on Windows.

Hi,  with python=3.9 ,ubuntu=20.04 , the same issue informs  and conda also failed to install  -c conda-forge or search -c pytorch.
% pip install fairseq==0.10.2
Collecting fairseq==0.10.2
Downloading fairseq-0.10.2.tar.gz (938 kB)
|████████████████████████████████| 938 kB 268 kB/s
Installing build dependencies ... done
Getting requirements to build wheel ... error
ERROR: Command errored out with exit status 1:
command: /opt/envs/ml/bin/python3.9 /opt/envs/ml/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpi9h33vpp
cwd: /tmp/pip-install-580ty_am/fairseq_9a2162ba544347ea84244836281ffd3e
Complete output (33 lines):
/tmp/pip-build-env-bwv17z0g/overlay/lib/python3.9/site-packages/setuptools/init.py:148: SetuptoolsDeprecationWarning: setup_requires is deprecated. Supply build dependencies using PEP 517 pyproject.toml build-requires.
warnings.warn(
Traceback (most recent call last):
File "setup.py", line 214, in
do_setup(package_data)
File "setup.py", line 136, in do_setup
setup(
File "/tmp/pip-build-env-bwv17z0g/overlay/lib/python3.9/site-packages/setuptools/init.py", line 158, in setup
_install_setup_requires(attrs)
File "/tmp/pip-build-env-bwv17z0g/overlay/lib/python3.9/site-packages/setuptools/init.py", line 153, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/tmp/pip-build-env-bwv17z0g/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 60, in fetch_build_eggs
raise SetupRequirementsError(specifier_list)
setuptools.build_meta.SetupRequirementsError: ['cython', 'numpy', 'setuptools>=18.0']
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/envs/ml/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in
main()
File "/opt/envs/ml/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/opt/envs/ml/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 130, in get_requires_for_build_wheel
return hook(config_settings)
File "/tmp/pip-build-env-bwv17z0g/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 154, in get_requires_for_build_wheel
return self._get_build_requires(
File "/tmp/pip-build-env-bwv17z0g/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 135, in _get_build_requires
self.run_setup()
File "/tmp/pip-build-env-bwv17z0g/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 150, in run_setup
exec(compile(code, file, 'exec'), locals())
File "setup.py", line 217, in
os.unlink(fairseq_examples)
IsADirectoryError: [Errno 21] Is a directory: 'fairseq/examples'
WARNING: Discarding https://files.pythonhosted.org/packages/02/59/1f466d82d64482e2809a167e0067979c23057b3752e668537d5ff24453c9/fairseq-0.10.2.tar.gz#sha256=45b90d8ccc3f5a4623a523fe2d0465f54413d03fd1ec9a9d7af0461148ca1a68 (from https://pypi.org/simple/fairseq/). Command errored out with exit status 1: /opt/envs/ml/bin/python3.9 /opt/envs/ml/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py get_requires_for_build_wheel /tmp/tmpi9h33vpp Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement fairseq==0.10.2 (from versions: 0.6.1, 0.6.2, 0.7.1, 0.7.2, 0.8.0, 0.9.0, 0.10.0, 0.10.1, 0.10.2)
ERROR: No matching distribution found for fairseq==0.10.2

has this been solved?

It's not just a problem with python 3.9. It appears even for 3.8, 3.7, 3.6.

I tried it with python 3.10.1 and 3.9.9 which yielded the same error as the OP. But going down to python 3.8.12 helped.
To be more precise this the dockerfile of the successful install:
FROM python:3.8.12-slim-bullseye
RUN apt-get -y update
RUN apt-get -y install build-essential=12.9
RUN pip3 install torch==1.10.1+cpu torchvision==0.11.2+cpu torchaudio==0.10.1+cpu -f https://download.pytorch.org/whl/cpu/torch_stable.html
RUN pip install -Iv fairseq==0.10.2

The same issue is happening in our CI test. https://github.com/espnet/espnet/runs/5011281760?check_suite_focus=true But it's interesting that the issue just come out in a very recent time. It does not have the issue from the point when the issue is raised.

I met the same problem on Windows10 python3.9.7. Why this problem has not been tackled yet?

For conda users, there are a few open PRs aiming at fixing this. In particular this one seems to work.
Until they get accepted, a workaround is to use sdvillal::fairseq as a dependency in your environment (linux and macos only).

For me (on MacOS 10.15, Python 3.9) the issue turned out to be the pip version: using pip 21.3.1 works fine but upgrading to pip 22.0.4 resulted in the above error

pip 21.3.1 did not work for me the only thing it did differently was it persevered when installing 0.10.2 and 0.10.1 failed and ended up successfully installing 0.10.0.

it does work on fairseq version:1.0.0, the solution is as follow:
git clone https://github.com/pytorch/fairseq
cd fairseq

activate env(that you created existed)
pip install --editable .

reference : https://stackoverflow.com/questions/65445611/cannot-install-fairseq-using-anaconda

still getting this buggo with 3.9.9 on Void Linux, but was able to run pip install fairseq=0.10.0

I was facing this issue on Google Colab; not on it's default python version, which was 3.7.13 but on version 3.6.9. I had changed python version via command sudo update-alternatives --config python3. Afterwards I needed to installed and updated pip via command sudo apt install python3-pip && python -m pip install --upgrade pip. The following is the error I was facing.
>>> pip install fairseq==0.9.0
Collecting fairseq==0.9.0
  Downloading fairseq-0.9.0.tar.gz (306 kB)
     |████████████████████████████████| 306 kB 4.7 MB/s
  Preparing metadata (setup.py) ... error
WARNING: Discarding https://files.pythonhosted.org/packages/67/bf/de299e082e7af010d35162cb9a185dc6c17db71624590f2f379aeb2519ff/fairseq-0.9.0.tar.gz#sha256=61206358b79f325ea0b46cfd8c95cdb81bfbcfb43cf12b47d1d5124ce7321d3b (from https://pypi.org/simple/fairseq/). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Could not find a version that satisfies the requirement fairseq==0.9.0 (from versions: 0.6.1, 0.6.2, 0.7.1, 0.7.2, 0.8.0, 0.9.0, 0.10.0, 0.10.1, 0.10.2)
ERROR: No matching distribution found for fairseq==0.9.0

Surprisingly error was resolved by updating setuptools via command pip3 install --upgrade setuptools

still getting this buggo with python 3.9.9 (pip 22.1) on Void Linux for fairseq versions 0.10.1 and 0.10.2, but was able to run pip install fairseq==0.10.0

yes, it also works with python 3.10 on windows11.

ERROR: Cannot install fairseq2==0.1.0 and fairseq2==0.1.1 because these package versions have conflicting dependencies.

I encountered the same issue while trying to install fairseq=0.10.2 using pip, and after several attempts that all ended in failure, I finally found a solution that worked for me (on Windows with Python 3.8).
Here's how I managed to install fairseq version 0.10.2:
1.You can download either the zip or tar.gz of the specific release directly from the https://github.com/facebookresearch/fairseq/releases/tag/v0.10.2
2.After downloading and extracting the file, I opened a command prompt with administrator privileges.
3.Navigated to the extracted fairseq directory: cd path\to\fairseq
4.pip install --editable ./
This method proved successful.
Please note that fairseq=0.10.2 requires PyTorch version >= 1.5.0 and Python version >= 3.6. Ensure these requirements are met prior to installation.
Hope this helps anyone else facing the same challenge!


Support for multiple ref seqs / chromosomes

Tooling should accomodate multiple ref chromosomes for panels. Write tests against an e.g. quick lab panel.

Added in v1a primaschema (unreleased)


Add additional label for prometheus metrics

Would it be possible to add the repository name to the prometheus metrics or have additional metrics for repo interaction? We're looking to start tracking usage at the repository level and currently the only way would be to separately store and parse the application logs.

Hi @tvanderwal-lmco , I think it's a valuable addition. 2 small concerns that shouldn't block this:

We hear users who are struggling with the cardinality of the metrics. lakeFS doesn't limit the amount of repos in it, so adding another label with unbounded amount of values can be tricky to manage. Users can solve this by dropping this label in the scrape config.
I think that would be the first time in lakeFS where prometheus collects user inserted data. Imagine a user which doesn't have access to list all the repos but have access to the metrics. We can hash the values to reduce some of the concern, but not all of it (number of repos for example)


Add ability to block/wait until standalone build succeeds or fails

@fson - Per our discussion to merge some of exptool functionality with exp!

This is awesome! Thanks for the pull request @mglagola.
Before starting to wait, we could show the "You can monitor the build at..." so people can go view the logs while waiting.
@anp had an idea that we could make this the default now, since it's almost always what you'd want. We can have --no-wait, which would revert back to the old behavior (exit once the build has started).

@fson Lmk what you think, added simpleSpinner, changed --wait to --no-wait and made sure You can monitor the build at... is always logged

Merged. Thanks again @mglagola, this makes the build command so much nicer!


Vacuum process fails if a table in the configuration file does not exist

In the scenario where the vacuum tool is executed with a configuration file that contains tables A, B, and C and table C does not exist; the tool fails.
Unfortunately, it fails half-way through the vacuum process, which is quite annoying.
Ideally, it should perform any of the following options:
a. Validate all tables in the configuration file before the execution begins, and exit if one of them is missing; or
b. Print a warning when it tries to vacuum the non-existent table, and then issue a summary towards the end of the execution with the results.
I'd prefer option b.
Regards,
–Jose

FWIW, option (b) would be more in line with how Circus Train works and we could do something like it does which then has return codes as described here: https://github.com/HotelsDotCom/circus-train/#exit-codes (replace the word "replications" with "housekeeping runs". We can probably even base the code required here on it.

Adding return codes to option (b) is a bonus and a good idea. Good call.

I've added explicit validation which as a side effect affects this issue: see PR https://github.com/HotelsDotCom/housekeeping/pull/39
So from a user if you specify a non existent table. The validation will fail (all tables are validated) and you are prompted to fix your config. None of the vacuum will run.
I am a bit concerned if we just report a failure in the logs and continue it will never be found and subsequent return codes will probably ignore as well as it looks like it is running. imo better to fail fast and make sure the config is correct before running.
thoughts?

When I raised the issue, I said I preferred option b. Actually, I prefer option a, as I completely agree with you that failing fast is better. No need to wait until the end and check any return codes and see what happened with the run.

The current validation does this table check as an implicit side-effect, if you turned the table validation off then it wouldn't catch this. My suggestion is we do this:

We have the boolean to control table validation and we clearly state in the documentation that it does two things - (1) checks that each table exists, (2) checks that the table contains the properties. If a user just wants the "does table exist" check they can set the value for the properties to "".
When the validation occurs it first checks if the table exists - if not then it logs a message saying the table doesn't exist (as opposed to the message saying that the property doesn't exist).

This way the user can control whether they want to do just a "table existence" check (boolean is true, property is empty string), or a "table property check" (boolean is true, property is default or some string)  or do no checks at all (set the boolean to false).

no, it will always check if the tables exist even if you list no properties to check
We don't have a boolean to control or turn on the table validation it is alway on.

OK, so you always get (1) then. That's fine. And if you want that but don't want to check any properties what do you do? Provide a list with one empty string?

you can overwrite with:
tables-validation:
  hive-table-properties:

fixed in PR #39


MIT

Would like the core under MIT licence? Could you consider splitting examples tests into other repositories?

Nope, my code is generally always licensed as Apache 2. It is still extremely permissive, with the addition over MIT of requiring you to state any changes you make to the code, protecting the name of the code, and protecting against patent trolls. Not that I expect this to ever get popular enough to need any of that, but I think it's the most fair & permissive open source license available.
Why would I split the samples and and tests into a different repository? They are specific to this codebase, so they should stay with the codebase. They also hardly clutter the repository.

Well the examples use three, was thinking it might be interesting to try a Kha example, to me three is just externs and js specific so does not really highlight the power of such libraries in Haxe.  If the examples are in another repository or abstracted out then the core would be added to haxelib to just be relevant algorithms, and have examples only on git to help with usage.  For instance that's the way I went with hxSpiro, it means the library does not assume too much about it's haxe visual use, also trying with a few approaches will help check it's flexibility.  I used to license BSD-2 but seems many haxelibs are MIT and the people who seem to need Apache2 are Adobe, Fb and corporations who halfheartedly go into opensource but maybe your reasoning is good.  It's just a bit nightmare with so many licenses around thought MIT was becoming common in haxe ecosystem.

Ahh, the library doesn't require JS or three.js or anything. The samples are for showing it in use, and for this repo they're largely for the demos in my blog posts (ex: http://hamaluik.com/posts/building-a-collision-engine-part-3-3d-gjk-collision-detection/#demo). Unfortunately, to actually show anything useful each sample is like 95% "other code", and only a couple lines showing the library in action.
When I 1.0 the library I'll make sure there are more samples (including standalone, Kha, Luxe, OpenFL, etc)—for now I'm focused on getting the base functionality to work.
Yea, most Haxelibs are MIT but I don't think MIT gives enough protections to open source, and Haxe is definitely compatible with Apache2. Many companies use Apache2 when dealing with open source because they have to by the nature of doing business (protecting trademarks and patents, which MIT does not, and including legal specificity). I get that Haxe is largely European and things may differ over there, but where I am Apache2 just makes sense.


[C++][CI] Typo in build_orc CMake macro

Describe the bug, including details regarding any error messages, version, and platform.
I've found a typo in the build_orc macro in the ThirdPartyToolchain, which means that the orc build command defaults to downloading zstd, instead of using the same configuration as the main arrow build command.
"-DZSTD_HOME=${ORZ_ZSTD_ROOT}") (From here)
Component(s)
C++, Continuous Integration

take


Feature: data streams

Description
[Describe what this change achieves]
Issues Resolved
[List any issues this PR will resolve]
Check List

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #605 (de0edd3) into 2.x (307c986) will increase coverage by 0.12%.
The diff coverage is n/a.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more
@@            Coverage Diff             @@
##              2.x     #605      +/-   ##
==========================================
+ Coverage   59.67%   59.79%   +0.12%
==========================================
  Files         268      282      +14
  Lines        8863     9198     +335
  Branches     1618     1698      +80
==========================================
+ Hits         5289     5500     +211
- Misses       3180     3289     +109
- Partials      394      409      +15

Impacted Files
Coverage Δ

...ement-dashboards-plugin/public/pages/Main/Main.tsx
5.88% <0.00%> (-0.45%)
:arrow_down:

...gement-dashboards-plugin/public/utils/constants.ts
95.23% <0.00%> (ø)

...eateIndex/components/MappingLabel/MappingLabel.tsx

...ages/CreateIndex/components/IndexMapping/helper.ts

...eateIndex/components/IndexMapping/IndexMapping.tsx

...rds-plugin/public/pages/CreateDataStream/hooks.tsx
100.00% <0.00%> (ø)

...m/containers/DefineDataStream/DefineDataStream.tsx
48.27% <0.00%> (ø)

...teDataStream/containers/DataStreamDetail/hooks.tsx
70.00% <0.00%> (ø)

...tream/containers/BackingIndices/BackingIndices.tsx
71.42% <0.00%> (ø)

.../DeleteDataStreamsModal/DeleteDataStreamsModal.tsx
83.33% <0.00%> (ø)

... and 12 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@Hailong-am For copyright, will add that in another branch after all the dev branches are merged.

Replaced label v2.6.0 with v2.x due to the 2.6.0 release code cut off.


HTTP Error 404: Not Found

Environment

System OS: Windows 10

Python Version: 3.90

Install Source: pip install spotDL

Version OR Commit hash: 3.6.2

Expected Behaviour
Download Music
Actual Behaviour
HTTP Error 404
Steps to Reproduce

spotdl https://open.spotify.com/track/0VjIjW4GlUZAMYd2vXMi3b?si=edf61be3e78148e5

Traceback
Fetching Song...
Gathering Spotify Metadata for:
https://open.spotify.com/track/0VjIjW4GlUZAMYd2vXMi3b?si=TNiemvONQviXQpWPSiR2Gw
Found YouTube URL for "The Weeknd - Blinding Lights" : https://www.youtube.com/watch?v=2ru92T7Y5z0

Error: HTTP Error 404: Not Found        While Error : The Weeknd - Blinding Lights
 Traceback (most recent call last):
  File "c:\users\devan\appdata\local\programs\python\python39\lib\site-packages\spotdl\download\downloa
der.py", line 261, in download_song
    youtubeHandler = YouTube(
  File "C:\Users\devan\AppData\Roaming\Python\Python39\site-packages\pytube\__main__.py", line 102, in
__init__
    self.prefetch()
  File "C:\Users\devan\AppData\Roaming\Python\Python39\site-packages\pytube\__main__.py", line 201, in
prefetch
    self.vid_info_raw = request.get(self.vid_info_url)
  File "C:\Users\devan\AppData\Roaming\Python\Python39\site-packages\pytube\request.py", line 43, in
get
    return _execute_request(url, headers=extra_headers).read().decode("utf-8")
  File "C:\Users\devan\AppData\Roaming\Python\Python39\site-packages\pytube\request.py", line 27, in
_execute_request
    return urlopen(request)  # nosec
  File "c:\users\devan\appdata\local\programs\python\python39\lib\urllib\request.py", line 214, in
urlopen
    return opener.open(url, data, timeout)
  File "c:\users\devan\appdata\local\programs\python\python39\lib\urllib\request.py", line 523, in open
    response = meth(req, response)
  File "c:\users\devan\appdata\local\programs\python\python39\lib\urllib\request.py", line 632, in
http_response
    response = self.parent.error(
  File "c:\users\devan\appdata\local\programs\python\python39\lib\urllib\request.py", line 555, in
error
    result = self._call_chain(*args)
  File "c:\users\devan\appdata\local\programs\python\python39\lib\urllib\request.py", line 494, in
_call_chain
    result = func(*args)
  File "c:\users\devan\appdata\local\programs\python\python39\lib\urllib\request.py", line 747, in
http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "c:\users\devan\appdata\local\programs\python\python39\lib\urllib\request.py", line 523, in open
    response = meth(req, response)
  File "c:\users\devan\appdata\local\programs\python\python39\lib\urllib\request.py", line 632, in
http_response
    response = self.parent.error(
  File "c:\users\devan\appdata\local\programs\python\python39\lib\urllib\request.py", line 561, in
error
    return self._call_chain(*args)
  File "c:\users\devan\appdata\local\programs\python\python39\lib\urllib\request.py", line 494, in
_call_chain
    result = func(*args)
  File "c:\users\devan\appdata\local\programs\python\python39\lib\urllib\request.py", line 641, in
http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 404: Not Found

Any other information:
@Silverarmor asked me to perform a full update using pip, did that and the HTTP Error 404 still persisted.

Same issue here. Previously it was working smoothly.

Same for me. urllib.error.HTTPError: HTTP Error 404: Not Found

@mpariseatsorint @V8gaming
check if you are using latest pytube version python -m pip install pytube --upgrade --force

@mpariseatsorint @V8gaming
check if you are using latest pytube version python -m pip install pytube --upgrade --force

still getting the error

@mpariseatsorint @V8gaming
check if you are using latest pytube version python -m pip install pytube --upgrade --force

This works. Thanks!

@mpariseatsorint @V8gaming
check if you are using latest pytube version python -m pip install pytube --upgrade --force

This indeed works!

Closing.

Closing.

still isn't working for me

@devangsaraogi Please try reinstall python completely as other users do not encounter this error anymore.


Hunter/cmakepp might be interesting for you, too

Not sure if you know these tool already:

https://github.com/toeb/cmakepp
https://github.com/ruslo/hunter
https://github.com/iauns/cpm
http://www.biicode.com

All of them have implemented some sort of external dependencies for cmake. It might be useful to try to model fips around the meta packages of some other tools to reuse the packages from there.
Not sure if this is feasable, but it might be worth a look.

Thanks! I stumbled over biicode recently, will definitely look at the others.

There's also this, might be a fancier option for fips' external dependency management:
https://github.com/buildinspace/peru

Another one: https://conan.io/

I've used this at places, more notably designed around the needs for mid/large VFX studios:
https://github.com/nerdvegas/rez
It integrates with CMake, but its strengths are more or less for building and deploying explicit versions of software packages and resolving environment variables to point to those explicit versions, with dependency resolution. It's great as well when you use particular software like Maya or Nuke with a C++ plugin compiled to a certain version of the software.

Interesting, thanks a lot for the pointer, I'll have a look :)


SetDoNotCacheHeaders overwrites headers that are more restrictive

If a user has set the Cache-Control header for a page, the form tag helper will overwrite the headers. In some cases this is appropriate, but if the Cache-Control header specifies a restriction such as no-store or must-revalidate, the antiforgery system wipes this out. This can open security holes in an application. Here is the relevant code as currently implemented:
    private void SetDoNotCacheHeaders(HttpContext httpContext)
    {
        // Since antifogery token generation is not very obvious to the end users (ex: MVC's form tag generates them
        // by default), log a warning to let users know of the change in behavior to any cache headers they might
        // have set explicitly.
        LogCacheHeaderOverrideWarning(httpContext.Response);

        httpContext.Response.Headers[HeaderNames.CacheControl] = "no-cache";
        httpContext.Response.Headers[HeaderNames.Pragma] = "no-cache";
    }

This should be changed to check the header before overwriting it. If no-cache is already present in the header, the header should be left alone. For more information see https://openidauthority.com/asp-net-core-mvc-anti-forgery-system-opens-security-hole/

@Eilon we should address this in 2.1, by splitting the cache control header apart, checking for no-cache and no-store, and appending them to the existing header (if any) rather than just blowing the header away. Forcing the Pragma header is fine, because it only has one value anyway :)

This issue was moved to aspnet/Home#2414


Noted that developer config settings require restart

Release Note
Updated System Console descriptions for Environment > Developer configuration settings in the System Console to clarify that changes require a server restart to take effect.

@hmhealey I'm not sure why parts of this PR are failing. Are you open to help?

I'd assume it's because this is based off an older version of the web app. Those should be fixed by /update-branch.
I'm also not sure why I assigned this back to you instead of just merging this

I just updated this since I missed before that we weren't updating the i18n files as well so the changes wouldn't have been actually visible in the app. Once these tests finish, I'll merge this


@conda-forge-admin,please add bot automerge

Hi! This is the friendly automated conda-forge-webservice.
I just wanted to let you know that I added bot automerge in conda-forge/ipyvizzu-feedstock#3.


SSR doesn't render HTML

When viewing the source code, I don't see the rendered HTML, but just the JavaScript variable containing the Prismic response.

some issue - seems that ssr is not configured correctly


Method to get $data

Hello Adam,
I propose this small change to make it possible to get data object. With this feature, we no longer have to make a child class of BoundData in https://github.com/Propaganistas/Laravel-Translatable-Bootforms package.

Coverage decreased (-0.2%) to 94.66% when pulling 1c20057faad4ffd9300d46ce2742d4794078e130 on sdebacker:master into 573a4d62f2f9dbdc4e0f173c1c7ded4ca3ccebba on adamwathan:master.

Looking at the Translatable Bootforms package to try and understand how this is being used in case there's a better solution, where is this translate() method defined? Is it expecting that you only bind models that implement the translate method?
https://github.com/Propaganistas/Laravel-Translatable-Bootforms/blob/master/src/Form/FormBuilder.php#L44

Also I think I would rather just add a data method like in the other package instead of making the first param optional in get. If you can make that change I'll merge this no biggie.

The translate method is here : https://github.com/dimsav/laravel-translatable/blob/master/src/Translatable/Translatable.php
I just created a data method. Thank you.

After the version bumping, I will make a PR on Propaganistas’ package.

Coverage decreased (-0.2%) to 94.647% when pulling 246c361aa98bc7754a7f3e14a0158577f1eeb3ec on sdebacker:master into 573a4d62f2f9dbdc4e0f173c1c7ded4ca3ccebba on adamwathan:master.

Hello Adam, do you plan to release a v0.8.8 version that include this new data() method?

Whoops, sorry! Done!

Nice & Thanks.


Add townhall for Sep 8.

Signed-off-by: Daniel Krook<EMAIL_ADDRESS>Describe at a high level the solution you're providing
Add today's townhall recording.

Thank you @krook


Rocket permissions and commands will fail completly when the Permissions.xml cant be parsed

So someone asked me yesterday for help with his rocket server, because no command was working anymore. Doesn't matter which group, admin or not admin, vanilla or rocket command etc, just no output. I looked at the log file, and it was full with exceptions from the Rocket permissions namespace, because of a single XML parse exception at the Permissions.xml.
My suggestion is to backup the permissions file and regenerate it automatically when it fails to load, or show mulitple big red warnings that the permissions file is corrupted. Or maybe just load the default permissions and dont touch the file at all.

All those actions make people wonder why they loose their permission, don't know if that's such a good idea.  If you read the error carefully you can figure what file is broken, and what line the xml is malformed.

Personally, I believe this isn't an issue we should be too worried about because, as @fr34kyn01535 said above, it's an easy fix on our end and would likely take quite a bit of work to work-around.

There is not mutch work to accomplish it, i am just worried that replacing it with the default file is not what people expect to happen. And just using the default config would be even more confusing.

You could format a warning message for failed permissions lookups from a malformed permissions xml, and if you can catch the part of the original exception that details where the syntax error is in the xml you could also print that into the warning message.
You'd still need to restart the server after fixing the permissions file though.


Is this project active?

Hi all.
I want to help, but I want to know if this project is active. Is this project active?

It depends on how you define "active." Currently, new PRs and issues will be accepted, but there is no active search for IoT resources.


Add a red button style

Can we get support for a .btn--red class? I think it would go great with the other buttons available.

Absolutely, I'll be able to add this in tomorrow.

Red buttons released in version 2.2.0.


Rp1

Hello!
Our team is thrilled and pleasantly surprised by your innovative Pull Request – the idea of integrating Camel's AI society and misalignment concepts into ChatDev is fascinating. However, given the considerable scale of these changes, we have concluded that we're expected to reference your repository link after testing. In the meantime, we suggest that you enhance the readme for this feature to facilitate its use by others.
In any case, we sincerely appreciate your interest in and support for ChatDev, and we welcome ongoing communication with you. ☺


Run instrumentation tests in Github Action and prepare tests to be run

In parallel of Java tests, instrumentation tests should be run on a
single emulator. If emulator crashes, logcat should be available as a
temporary build artifact. If test fails, test report should be available
as a temporary build artifact
As this is totally new, some tests are expected to fail. Easy-to-fix tests might be fixed. Others might be ignored for now a fixed later in EE-1180.
JIRA: EE-1165

Without the first commit on CSM, the tests crashes in the middle of the run. It is because, even when we recovered from Tape queue corruption, an exception was thrown in tests. And sometimes the exception was thrown outside the test thread, which caused the whole test suit to stop.
The last commit on native, is a quick fix for functional tests on Advanced Native (We were aware of this issue but as it was green on our laptop, we didn't take times for this). If you prefer, I can remove this and investigate more in EE-1180.

My concern is that I've started this review which was about adding Android Tests, and that I now see unrelated commits appearing in the PR, changing the scope of the review. In a former project we used to Squash & Merge PR, having in the end a linear commit history like on Gerrit. But that implied a PR to be like a commit, atomic, maybe I'm biased because of this.

In any cases, commits such as fixing CSM are unrelated from the PR description, so I think you should change PR title/description.

When OK to merge, I will remove the expectation on the old build job, and and expectations on the new java-tests and android-test.

@TurpIF Agreed with your approach. Let's make it mandatory for merging first and see how far we can go with that.


Fix: Add logging path modification for OpenAI usage

This is to ensure that extraneous files are created in specified folder.
Refer to #211

Hmm, I see. What are your opinions of using an env_var here: either to prefix the logging files to a specific folder, or to enable/disable?
I was thinking maybe something like DSPY_LOGGING_DIR to prefix a folder to save logs, or DSPY_LOGGING to enable/disable logging.

@okhat I updated the PR to focus on just giving the user the ability to specify a folder to save logs to — the mechanism to allow enabling/disabling of logging seems more complicated, but this should provide some level on configurability on where logs are stored.

I've added an initial global logger, with configuration here: #684.
Does the API offered in that PR, cover all the uses outlined here?

I've added an initial global logger, with configuration here: #684. Does the API offered in that PR, cover all the uses outlined here?

Yup, looks perfect — thanks so much!


Fixed - Theme Check: Issue of 0 files being inspected.

What are you adding in this PR?
Fixes shopify/cli#3497
The main issue was that on windows, no liquid or json files were being selected despite '**/*.{liquid,json}' as the glob function.
this happens because in windows, the path separator output by path.join is '' which conflicts with the glob escape character.
To solve it, have normalized the path and just replaced '' with  '/' . This should solve this since it wont have any impact on existing linux paths.
What's next? Any followup issues?
N/A -
What did you learn?

I have signed the CLA!

You are amazing. This was on my TODO list but couldn't get around doing it with everything else on there. Thank you!
I'll add a changeset for a patch bump and see that this gets into the CLI ASAP.
🙏


[GetConnect] MultipartFile read from filepath

Hi,
Is the MultipartFile able to read file from filepath? If not, can this be added as a feature?
Btw, I tried to use the code from dio to read the file from filepath, however, this seems not possible since the Multipart in Getx is not taking stream for bytes. Can we have a constructor that takes stream as argument?
Thanks.

Use File(filepath).readAsBytesSync() to get bytes from file.
Getting a path file may be supported in the future, but I don't like this feature personally, since it only works on a mobile device.
Sending bytes is a universal solution, and has automatic integration with Imagepicker, where pickedFile delivers the file bytes directly, but in the future I can add the fromFile and fromString method for those who do not want to have their application running on the web, only on mobile.

Thanks for the advice, this should works for me.
Btw, I found that the FormData is not really working with GetConnect, I am trying to migrate from existing dio with GetConnect, get stuck with those required FormData.
Hope these issues are fixed as soon as possible, so we can have more time to test our app with GetConnect implementation, due to the FormData issue, we have to still stick with Dio.

Thanks for the advice, this should works for me.
Btw, I found that the FormData is not really working with GetConnect, I am trying to migrate from existing dio with GetConnect, get stuck with those required FormData.
Hope these issues are fixed as soon as possible, so we can have more time to test our app with GetConnect implementation, due to the FormData issue, we have to still stick with Dio.

I am using the latest version in a application real with multipart.
Can you confirm that this works for you?
I ended up simplifying the api, and for me it works well, but I would like to know if other people are experiencing any problems with multipart request.

Added on 3.22

The FormData issues are fixed with the latest version, however, we encountered other errors, will report the issue if confirmed it is a bug in the package. Thanks!


Distance based DPI rendering

... i just had the idea that if you are in a certan render distance to move to hight dpi ...
.... so you can let it render sutff with less pixel if you are far away and if you have a realy close look you can go hight dpi and there should be no pixels ...
-> for flatland & wayland -> dynamically applay fractual scaling
... but the important part is to have an upper and lower bound ...
sorce: https://discord.com/channels/647222689777582080/659858947896770590/1276321359517061193

https://wayland.app/protocols/fractional-scale-v1


CharacterTextSplitter with keep_separator=True sets the separator to the beginning of each chunk instead of an end

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain rather than my code.
[X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code
from langchain.text_splitter import CharacterTextSplitter

splitter = CharacterTextSplitter(
    chunk_size=10,
    chunk_overlap=0,
    separator=". ",
    keep_separator=True,
)

for d in splitter.create_documents(["Text 1. Text 2. Text 3. Text 4."]):
    print(d.page_content)
    print("---")

Error Message and Stack Trace (if applicable)
Text 1
---
. Text 2
---
. Text 3
---
. Text 4.
---

Description
I'm trying to split text by sentence, while keeping end-of-sentence punctuation. Instead of putting the punctuation back at the end of the corresponding chunk, the library adds it to the front of the following chunk.
This problem is quite critical if the output is used for text-to-speech input.
System Info
langchain==0.1.16
langchain-community==0.0.34
langchain-core==0.1.46
langchain-text-splitters==0.0.1

MacOS, Python 3.10.13

Hi @VPetukhov , can I work on this issue ?


MineDojo error Running

max6468@max6468-MacBookPro:~/Escritorio$ vglrun python3 ia.py
[INFO:minedojo.tasks] Loaded 1572 Programmatic tasks, 1558 Creative tasks, and 1 special task: "Playthrough". Totally 3131 tasks loaded.
/home/max6468/.local/lib/python3.10/site-packages/numpy/core/numeric.py:330: RuntimeWarning: invalid value encountered in cast
multiarray.copyto(a, fill_value, casting='unsafe')
/home/max6468/.local/lib/python3.10/site-packages/gym/spaces/box.py:73: UserWarning: WARN: Box bound precision lowered by casting to float32
logger.warn(
[INFO:minedojo.tasks] Loaded 1572 Programmatic tasks, 1558 Creative tasks, and 1 special task: "Playthrough". Totally 3131 tasks loaded.
Traceback (most recent call last):
File "/home/max6468/Escritorio/ia.py", line 7, in
obs = env.reset()
File "/home/max6468/.local/lib/python3.10/site-packages/gym/core.py", line 292, in reset
return self.env.reset(**kwargs)
File "/home/max6468/.local/lib/python3.10/site-packages/minedojo/sim/wrappers/ar_nn/delta_inventory_wrapper.py", line 78, in reset
observation = self.env.reset(**kwargs)
File "/home/max6468/.local/lib/python3.10/site-packages/gym/core.py", line 319, in reset
observation = self.env.reset(**kwargs)
File "/home/max6468/.local/lib/python3.10/site-packages/minedojo/sim/wrappers/ar_nn/nn_action_space_wrapper.py", line 259, in reset
obs = self.env.reset(**kwargs)
File "/home/max6468/.local/lib/python3.10/site-packages/minedojo/tasks/meta/base.py", line 87, in reset
obs = self.env.reset()
File "/home/max6468/.local/lib/python3.10/site-packages/minedojo/sim/wrappers/fast_reset.py", line 71, in reset
return self.env.reset()
File "/home/max6468/.local/lib/python3.10/site-packages/minedojo/sim/sim.py", line 419, in reset
raw_obs = self._bridge_env.reset(episode_id, [xml])[0]
File "/home/max6468/.local/lib/python3.10/site-packages/minedojo/sim/bridge/bridge_env/bridge_env.py", line 72, in reset
self._setup_instances()
File "/home/max6468/.local/lib/python3.10/site-packages/minedojo/sim/bridge/bridge_env/bridge_env.py", line 157, in _setup_instances
self._instances.extend([f.result() for f in instance_futures])
File "/home/max6468/.local/lib/python3.10/site-packages/minedojo/sim/bridge/bridge_env/bridge_env.py", line 157, in
self._instances.extend([f.result() for f in instance_futures])
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 451, in result
return self.__get_result()
File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
raise self._exception
File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/max6468/.local/lib/python3.10/site-packages/minedojo/sim/bridge/bridge_env/bridge_env.py", line 177, in _get_new_instance
instance.launch(replaceable=self._is_fault_tolerant)
File "/home/max6468/.local/lib/python3.10/site-packages/minedojo/sim/bridge/mc_instance/instance.py", line 201, in launch
raise EOFError(
EOFError: usage: /tmp/tmpafkhs468/Minecraft/launchClient.sh [-replaceable] [-port 10000] [-scorepolicy 0123] [-env]
Minecraft process finished unexpectedly. There was an error with Malmo.

same error


[CVE-2017-12904] Remote code execution

Dear users,
Jeriko One discovered a vulnerability that allows a remote attacker to execute arbitrary code on your computer.
An attacker can craft an RSS item with shell code in the title and/or URL. When you bookmark such an item, your shell will execute that code. The vulnerability is triggered when bookmark-cmd is called; if you abort bookmarking before that, you're safe.
Newsbeuter versions 0.7 through 2.9 are affected.
Workaround
Update 2017.08.18: Do not use bookmarking until you apply the fix. See the comment below for details.
First of all, set bookmark-autopilot to no (that's the default.) This gives you a chance to review inputs before executing your bookmark-cmd.
Second, when bookmarking items, pay close attention to titles and URLs. I can't possibly teach you how to recognize shell code in just a few paragraphs, so if unsure, just don't bookmark the thing.
Resolution
A fix has already been pushed to our Git repository: https://github.com/akrennmair/newsbeuter/commit/96e9506ae9e252c548665152d1b8968297128307
I managed to get in touch with maintainers in AUR, Debian, FreeBSD and Gentoo, so if you're running one of those, an update should arrive soon. If you're running something else, I encourage you to find out who maintains Newsbeuter for your distribution, contact them and point to the aforementioned commit. They'll know what to do.
Call to security researchers
If you discover a vulnerability, please disclose it to me privately at<EMAIL_ADDRESS>preferably encrypting the message for PGP key 356961A20C8BFD03.
(This has also been posted on our mailing list).

This gives you a chance to review inputs before executing your bookmark-cmd.

This is not foolproof, the following PoC gives no indication that something is up unless you open the article first.

<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
  <title>Example feed</title>
  <link href="http://example.com/blog/"/>
  <link type="application/atom+xml" rel="self" href="http://example.com/blog/atom.xml"/>
  <updated>2016-01-01T00:00:00+00:00</updated>
  <id>http://test.com.com/blog/</id>
  <author>
    <name>Test feed</name>
  </author>
  <entry>
    <title>Example feed title                                                                                $(echo 'RCE' >> $HOME/exploit)</title>
    <link href="http://example.com/posts/test/html"/>
    <updated>2016-01-01T00:00:00+00:00</updated>
    <id>http://example.com/posts/test</id>
    <content type="html">
     Example content
    </content>
  </entry>
</feed>

I stand corrected. The original PoC by Jeriko One actually used the same technique, but i was using a different example in my testing and totally forgot about this. Thanks, @tsipinakis!
New workaround: Don't bookmark items.
I'm going to update the top post now, and also write to the mailing list.

FYI, the patch in https://github.com/akrennmair/newsbeuter/commit/96e9506ae9e252c548665152d1b8968297128307 doesn't apply to Newsbeuter 2.9, so the distros are writing their own patches.
This is what Debian is using in their Newsbeuter 2.9-5+deb9u1 package:
Description: Fix a RCE vulnerability in the bookmark command
 Newsbeuter didn't properly escape the title and description fields before
 passing them to the bookmarking program which could lead to remote code
 execution using the shells command substitution functionality (e.g. "$()", ``,
 etc)

Origin: upstream, https://github.com/akrennmair/newsbeuter/commit/96e9506ae9e252c548665152d1b8968297128307
Last-Update: 2017-08-18

--- newsbeuter-2.9.orig/src/controller.cpp
+++ newsbeuter-2.9/src/controller.cpp
@@ -1274,9 +1274,10 @@ std::string controller::bookmark(const s
        std::string bookmark_cmd = cfg.get_configvalue("bookmark-cmd");
        bool is_interactive = cfg.get_configvalue_as_bool("bookmark-interactive");
        if (bookmark_cmd.length() > 0) {
-               std::string cmdline = utils::strprintf("%s '%s' %s %s",
+               std::string cmdline = utils::strprintf("%s '%s' '%s' '%s'",
                                                       bookmark_cmd.c_str(), utils::replace_all(url,"'", "%27").c_str(),
-                                                      stfl::quote(title).c_str(), stfl::quote(description).c_str());
+                                                      utils::replace_all(title,"'", "%27").c_str(),
+                                                      utils::replace_all(description,"'", "%27").c_str());

                LOG(LOG_DEBUG, "controller::bookmark: cmd = %s", cmdline.c_str());

@lfam, I sent patches for 2.8 and 2.9 to all maintainers that responded to me—AUR, Debian, FreeBSD, Gentoo, and now Slackware. The one you quote above is mine.
I now realize I should've simply pushed them to the repo. So I did; see newsbeuter-2.8-cve-2017-12904 and newsbeuter-2.9-cve-2017-12904 branches.


angr dependencies: 9.0.5739 -> 9.0.5903

Motivation for this change
Update all angr dependencies to 9.0.5903
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 114046 at b13101cf run on x86_64-linux 1

12 packages built:

python37Packages.ailment
python37Packages.archinfo
python37Packages.claripy
python37Packages.pyvex
python38Packages.ailment
python38Packages.archinfo
python38Packages.claripy
python38Packages.pyvex
python39Packages.ailment
python39Packages.archinfo
python39Packages.claripy
python39Packages.pyvex

Either you change the wheel type to 2014 or disable the failing builds on aarch64.

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 114046 run on x86_64-darwin 1

6 packages failed to build and already failed to build on hydra master:

python37Packages.ailment: log was empty
python37Packages.pyvex: plain log  |  pretty log
python38Packages.ailment: log was empty
python38Packages.pyvex: plain log  |  pretty log
python39Packages.ailment: log was empty
python39Packages.pyvex: plain log  |  pretty log

6 packages built:

python37Packages.archinfo
python37Packages.claripy
python38Packages.archinfo
python38Packages.claripy
python39Packages.archinfo
python39Packages.claripy

The following issues got detected with the above build packages.
Please fix at least the ones listed with your changed packages:

python37Packages.ailment:

Add a checkPhase for tests, or at least pythonImportsCheck.
See: https://github.com/jtojnar/nixpkgs-hammering/blob/master/explanations/python-include-tests.md

python38Packages.ailment:

Add a checkPhase for tests, or at least pythonImportsCheck.
See: https://github.com/jtojnar/nixpkgs-hammering/blob/master/explanations/python-include-tests.md

python39Packages.ailment:

Add a checkPhase for tests, or at least pythonImportsCheck.
See: https://github.com/jtojnar/nixpkgs-hammering/blob/master/explanations/python-include-tests.md

Result of nixpkgs-review pr 114046 run on x86_64-linux 1

12 packages built:

python37Packages.ailment
python37Packages.archinfo
python37Packages.claripy
python37Packages.pyvex
python38Packages.ailment
python38Packages.archinfo
python38Packages.claripy
python38Packages.pyvex
python39Packages.ailment
python39Packages.archinfo
python39Packages.claripy
python39Packages.pyvex

Either you change the wheel type to 2014 or disable the failing builds on aarch64.

Disabled on aarch64.

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
If you have any questions or problems please reach out to SuperSandro2000 on IRC.
Result of nixpkgs-review pr 114046 run on x86_64-linux 1

12 packages built:

python37Packages.ailment
python37Packages.archinfo
python37Packages.claripy
python37Packages.pyvex
python38Packages.ailment
python38Packages.archinfo
python38Packages.claripy
python38Packages.pyvex
python39Packages.ailment
python39Packages.archinfo
python39Packages.claripy
python39Packages.pyvex

The following issues got detected with the above build packages.
Please fix at least the ones listed with your changed packages:

python37Packages.ailment:

++ /home/hotpi/src/nixpkgs-hammering/result/bin/nixpkgs-hammer --exclude attribute-ordering --exclude explicit-phases --exclude no-flags-array python37Packages.ailment
When evaluating attribute ‘python37Packages.ailment’:
warning: python-include-tests
Add a checkPhase for tests, or at least pythonImportsCheck.
See: https://github.com/jtojnar/nixpkgs-hammering/blob/master/explanations/python-include-tests.md

python38Packages.ailment:

++ /home/hotpi/src/nixpkgs-hammering/result/bin/nixpkgs-hammer --exclude attribute-ordering --exclude explicit-phases --exclude no-flags-array python38Packages.ailment
When evaluating attribute ‘python38Packages.ailment’:
warning: python-include-tests
Add a checkPhase for tests, or at least pythonImportsCheck.
See: https://github.com/jtojnar/nixpkgs-hammering/blob/master/explanations/python-include-tests.md

python39Packages.ailment:

++ /home/hotpi/src/nixpkgs-hammering/result/bin/nixpkgs-hammer --exclude attribute-ordering --exclude explicit-phases --exclude no-flags-array python39Packages.ailment
When evaluating attribute ‘python39Packages.ailment’:
warning: python-include-tests
Add a checkPhase for tests, or at least pythonImportsCheck.
See: https://github.com/jtojnar/nixpkgs-hammering/blob/master/explanations/python-include-tests.md

Result of nixpkgs-review pr 114046 run on x86_64-linux 1

12 packages built:

python37Packages.ailment
python37Packages.archinfo
python37Packages.claripy
python37Packages.pyvex
python38Packages.ailment
python38Packages.archinfo
python38Packages.claripy
python38Packages.pyvex
python39Packages.ailment
python39Packages.archinfo
python39Packages.claripy
python39Packages.pyvex


DOC: stats: Fix a reference for the genexpon distribution.

Gordon and Breach are the original publisher.  The publisher has changed over the years (Routledge, CRC Press, Taylor & Francis).

This can be seen in the preview of the book so I'll merge as a straightforward fix. Thanks @WarrenWeckesser !


[BACKLOG-15721] File add/rename fixes for file open/save

@ddiroma

Build Completed
:fire: This pull request has some issues. It would be preferable to fix them in order for it to be just perfect. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
mvn -B -fn -f pom.xml clean install && mvn -B -f pom.xml site

Cleanup Commands
mvn -B -f pom.xml build-helper:remove-project-artifact

Changed files
  plugins/file-open-save/core/src/main/java/org/pentaho/repo/controller/RepositoryBrowserController.java
  plugins/file-open-save/core/src/main/java/org/pentaho/repo/endpoint/RepositoryBrowserEndpoint.java
  plugins/file-open-save/core/src/main/java/org/pentaho/repo/extension/RepositoryOpenSaveExtensionPoint.java
  plugins/file-open-save/core/src/main/javascript/app/app.component.js
  plugins/file-open-save/core/src/main/javascript/app/app.html
  plugins/file-open-save/core/src/main/javascript/app/components/error/error.component.js
  plugins/file-open-save/core/src/main/javascript/app/components/error/error.html
  plugins/file-open-save/core/src/main/javascript/app/components/files/edit.directive.js
  plugins/file-open-save/core/src/main/javascript/app/components/files/files.component.js
  plugins/file-open-save/core/src/main/javascript/app/components/files/files.html
  plugins/file-open-save/core/src/main/javascript/app/components/folder/folder.component.js
  plugins/file-open-save/core/src/main/javascript/app/components/folder/folder.html
  plugins/file-open-save/core/src/main/resources/i18n/file-open-save/messages_en.properties

Unit test coverage change
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.
org.pentaho.di.job.entries.evalfilesmetrics.JobEntryEvalFilesMetrics

Branch Change: -.39%:small_red_triangle_down:
Complexity Change: -.51%:small_red_triangle_down:
Instruction Change: -.08%:small_red_triangle_down:

org.pentaho.di.job.entries.ftpsget.JobEntryFTPSGet

Branch Change:  + .53%
Complexity Change:  + .56%
Instruction Change:  + .07%

org.pentaho.di.job.entries.sftpput.JobEntrySFTPPUT

Branch Change:  + .58%
Complexity Change:  + .66%
Instruction Change:  + .09%

org.pentaho.di.job.entries.sql.JobEntrySQL

Instruction Change:  + 3.28%
Line Change:  + 3.87%

org.pentaho.di.trans.step.StepMeta

Branch Change:  + .68%
Complexity Change:  + .58%
Instruction Change:  + .19%
Line Change:  + .26%

org.pentaho.di.trans.steps.calculator.CalculatorMetaFunction

Branch Change: -8.70%:small_red_triangle_down:
Complexity Change: -3.03%:small_red_triangle_down:
Instruction Change: -.75%:small_red_triangle_down:
Line Change: -.89%:small_red_triangle_down:

org.pentaho.di.trans.steps.checksum.CheckSumMeta

Instruction Change:  + .11%

org.pentaho.di.trans.steps.combinationlookup.CombinationLookupMeta

Instruction Change:  + .04%

org.pentaho.di.trans.steps.datagrid.DataGridMeta

Branch Change:  + 1.92%
Complexity Change:  + 1.69%
Instruction Change:  + .17%

org.pentaho.di.trans.steps.dbproc.DBProcMeta

Instruction Change:  + .11%

org.pentaho.di.trans.steps.groupby.GroupByMeta

Branch Change: -1.04%:small_red_triangle_down:
Complexity Change: -1.05%:small_red_triangle_down:

org.pentaho.di.trans.steps.ifnull.IfNullMeta

Branch Change: -2.00%:small_red_triangle_down:
Complexity Change: -1.89%:small_red_triangle_down:
Instruction Change: -.17%:small_red_triangle_down:

org.pentaho.di.trans.steps.insertupdate.InsertUpdateMeta

Instruction Change:  + .05%

org.pentaho.di.trans.steps.ldapoutput.LDAPOutputMeta

Instruction Change:  + .06%

org.pentaho.di.trans.steps.memgroupby.MemoryGroupByMeta

Branch Change:  + 2.35%
Complexity Change:  + 2.70%
Instruction Change:  + .35%
Line Change:  + .97%

org.pentaho.di.trans.steps.replacestring.ReplaceStringMeta

Instruction Change:  + .07%

org.pentaho.di.trans.steps.rowgenerator.RowGeneratorMeta

Instruction Change:  + .08%

org.pentaho.di.trans.steps.sort.SortRowsMeta

Instruction Change: -.08%:small_red_triangle_down:

org.pentaho.di.trans.steps.uniquerows.UniqueRowsMeta

Instruction Change: -.18%:small_red_triangle_down:

org.pentaho.di.trans.steps.webservices.WebServiceMeta

Branch Change: -1.79%:small_red_triangle_down:
Complexity Change: -1.19%:small_red_triangle_down:
Instruction Change: -.07%:small_red_triangle_down:

org.pentaho.di.trans.steps.gpload.GPLoadMeta

Instruction Change:  + .13%

org.pentaho.di.trans.steps.salesforceinsert.SalesforceInsertMeta

Instruction Change: -.15%:small_red_triangle_down:

org.pentaho.di.trans.steps.salesforceupsert.SalesforceUpsertMeta

Instruction Change:  + .14%

Build Completed
:fire: This pull request has some issues. It would be preferable to fix them in order for it to be just perfect. See below for more details. Some links are also available below for further assistance in addressing those issues.
Build Commands
mvn -B -fn -f pom.xml clean install && mvn -B -f pom.xml site

Cleanup Commands
mvn -B -f pom.xml build-helper:remove-project-artifact

Changed files
  plugins/file-open-save/core/src/main/java/org/pentaho/repo/controller/RepositoryBrowserController.java
  plugins/file-open-save/core/src/main/java/org/pentaho/repo/endpoint/RepositoryBrowserEndpoint.java
  plugins/file-open-save/core/src/main/java/org/pentaho/repo/extension/RepositoryOpenSaveExtensionPoint.java
  plugins/file-open-save/core/src/main/javascript/app/app.component.js
  plugins/file-open-save/core/src/main/javascript/app/app.html
  plugins/file-open-save/core/src/main/javascript/app/components/error/error.component.js
  plugins/file-open-save/core/src/main/javascript/app/components/error/error.html
  plugins/file-open-save/core/src/main/javascript/app/components/files/edit.directive.js
  plugins/file-open-save/core/src/main/javascript/app/components/files/files.component.js
  plugins/file-open-save/core/src/main/javascript/app/components/files/files.html
  plugins/file-open-save/core/src/main/javascript/app/components/folder/folder.component.js
  plugins/file-open-save/core/src/main/javascript/app/components/folder/folder.html
  plugins/file-open-save/core/src/main/resources/i18n/file-open-save/messages_en.properties

Unit test coverage change
These statistics help you identify how your changes have affected the coverage of the following files. If a file is not in this list, then its coverage was not affected by your changes. To get some help interpreting these metrics, please refer to Jacoco's documentation.
org.pentaho.di.core.injection.bean.BeanInjector

Branch Change: -.84%:small_red_triangle_down:
Complexity Change: -1.39%:small_red_triangle_down:

org.pentaho.di.job.entries.checkdbconnection.JobEntryCheckDbConnections

Branch Change:  + 1.23%
Complexity Change:  + 1.47%
Instruction Change:  + .21%

org.pentaho.di.job.entries.evaluatetablecontent.JobEntryEvalTableContent

Branch Change:  + 1.01%
Complexity Change:  + 1.06%
Instruction Change:  + .17%

org.pentaho.di.job.entries.waitforsql.JobEntryWaitForSQL

Branch Change:  + .66%
Complexity Change:  + .93%
Instruction Change:  + .13%

org.pentaho.di.trans.steps.calculator.CalculatorMetaFunction

Branch Change:  + 8.70%
Complexity Change:  + 3.03%
Instruction Change:  + .75%
Line Change:  + .89%

org.pentaho.di.trans.steps.combinationlookup.CombinationLookupMeta

Instruction Change:  + .04%

org.pentaho.di.trans.steps.constant.ConstantMeta

Instruction Change:  + .10%

org.pentaho.di.trans.steps.csvinput.CsvInputMeta

Instruction Change:  + .05%

org.pentaho.di.trans.steps.groupby.GroupByMeta

Branch Change:  + 4.17%
Complexity Change:  + 4.21%
Instruction Change:  + .14%
Line Change:  + .37%

org.pentaho.di.trans.steps.insertupdate.InsertUpdateMeta

Instruction Change:  + .05%

org.pentaho.di.trans.steps.ldapoutput.LDAPOutputMeta

Instruction Change:  + .06%

org.pentaho.di.trans.steps.rowgenerator.RowGeneratorMeta

Instruction Change:  + .08%

org.pentaho.di.trans.steps.singlethreader.SingleThreaderMeta

Instruction Change: -.10%:small_red_triangle_down:

org.pentaho.di.trans.steps.sort.SortRowsMeta

Instruction Change: -.08%:small_red_triangle_down:

org.pentaho.di.trans.steps.sortedmerge.SortedMergeMeta

Instruction Change: -.18%:small_red_triangle_down:

org.pentaho.di.trans.steps.symmetriccrypto.symmetriccryptotrans.SymmetricCryptoTransMeta

Branch Change: -3.33%:small_red_triangle_down:
Complexity Change: -1.85%:small_red_triangle_down:
Instruction Change: -.28%:small_red_triangle_down:

org.pentaho.di.trans.steps.synchronizeaftermerge.SynchronizeAfterMergeMeta

Instruction Change: -.04%:small_red_triangle_down:

org.pentaho.di.trans.steps.textfileinput.TextFileInputMeta

Branch Change: -.53%:small_red_triangle_down:

org.pentaho.di.trans.steps.uniquerows.UniqueRowsMeta

Instruction Change:  + .18%

org.pentaho.di.trans.steps.googleanalytics.GaInputStepMeta

Instruction Change:  + .07%

org.pentaho.di.trans.steps.salesforceinsert.SalesforceInsertMeta

Instruction Change:  + .15%

org.pentaho.di.trans.steps.salesforceupdate.SalesforceUpdateMeta

Instruction Change: -.17%:small_red_triangle_down:


feat(dropdown): cva

👋 Thanks for creating a pull request! 🚀 Checkout the storybook we created for it here.


Fixed function to create components node if missing

Pull Request Description
What issue does this change request address? (Use "#" before the issue to link it, i.e., #42.)
#15
What are the significant changes in functionality due to this change request?
When a HERON XML script containing no <Components> node is input to the src/heron/create_componentsets_in_HERON() function, the function now adds a new <Components> node. It also prints a message stating that the addition was made. This mirrors the function's behavior when dealing with input scripts that are missing <economics> or <CashFlow> nodes.

For Change Control Board: Change Request Review
The following review must be completed by an authorized member of the Change Control Board.

[ ] 1. Review all computer code.
[ ] 2. Make sure the Python code and commenting standards are respected (camelBack, etc.) - See on the wiki for details.
[ ] 3. Automated Tests should pass, including run_tests, pylint, and manual building tests.
[ ] 4. If significant functionality is added, there must be tests added to check this. Tests should cover all possible options.  Multiple short tests are preferred over one large test.
[ ] 5. If the change modifies or adds a requirement or a requirement based test case, the Change Control Board's Chair or designee also needs to approve the change.  The requirements and the requirements test shall be in sync.
[ ] 6. The merge request must reference an issue.  If the issue is closed, the issue close checklist shall be done.

could you fix the typo on this line?
https://github.com/idaholab/FORCE/pull/16/files#diff-e5a05e2d7232d1a807b092ef316d5d19d9eaec4b6043a74cd4dd42512f90daa2R34

And I believe the multi-line strings like the input in this docstring should be indented at least to the comma on the previous line (it was bothering me and might as well ask since you're already editing the file!): https://github.com/idaholab/FORCE/pull/16/files#diff-e5a05e2d7232d1a807b092ef316d5d19d9eaec4b6043a74cd4dd42512f90daa2R36


NIFI-6859: Add scripted versions of RecordSinkService, RulesEngineService, and ActionHandler

Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
Added ScriptedRecordSink, ScriptedRulesEngine, and ScriptedActionHandler.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:

[x] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?

[x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.

[x] Has your PR been rebased against the latest commit within the target branch (typically master)?

[x] Is your initial contribution a single, squashed commit? Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not squash or use --force when pushing to allow for clean monitoring of changes.

For code changes:

[x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
[x] Have you written or updated unit tests to verify your changes?
[ ] Have you verified that the full build is successful on both JDK 8 and JDK 11?
[ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
[ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
[ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
[ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?

For documentation related changes:

[x] Have you ensured that format looks appropriate for the output in which it is rendered?

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

reviewing

@mattyb149 I was able to test different combinations of Scripted Implementations, especially within the context of the MetricsEventReportingTask which all worked as expected.  One issue I can only see in Travis is with some of test failing (locally test execution worked without issue for me).  Once Travis has been investigated/resolved I think we'll be good to +1 and merge.

LGTM +1
will merge shortly


logging removed #26 (todo: implement logging #27)

fix project structure #25
file naming #24

close #32


v2.8.2, without React deprecation warnings?

Version
2.8.1
Hi there! We maintain a yeoman generator at work, with many users and production apps still using React Router v2.8.1. Recently, in React 15.5.0 - it seems as if React.createClass and React.PropTypes have been moved, and deprecation warnings added for React 16.
https://facebook.github.io/react/blog/2017/04/07/react-v15.5.0.html
I know this is probably the very least of your priorities, but we'd greatly appreciate a patch release of React Router with changes to RouterContext.js as described in the migration guide provided by Facebook 😄
There's a drop-in shim for React.createClass, and the propTypes change is a simple pass through with sed
Our developers would greatly appreciate these warnings suppressed during development - since 15.5.0 is likely of the last v15 React releases and we'd be able to shrink-wrap the generator and move to v16 with React Router @4 in the future.
Until then, thanks either way!

You can use 3.0.5 safely with 2.x code. 3.0 is just 2.0 with the deprecation warnings (ours, not React's) removed. And 3.0.5 fixes for all the createClass and PropTypes deprecations.

Also, there is a 15.6 planned that should ease things a bit: https://github.com/facebook/react/issues/9398


NuGet targets portable-net45+wp8+win8+wpa81, wpa81, win81 collide when combining UAP and Portable Library into APPX

After upgrading to 1.0.1, I discovered that the Visual Studio won't build my Win8.1/WP8.1 package, because MetroLog.pdb from one folder collides with MetroLog.pdb from another folder in packages folder.
The error message follows:
C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0\AppxPackage\Microsoft.AppXPackage.Targets(1670,5): // repeating
error APPX1101: Payload contains two or more files with the same destination path 'MetroLog.pdb'. Source files:
error APPX1101: I:\Documents\Projekty\stupid things\MetroLogDepApp\packages\MetroLog.1.0.1\lib\win81\MetroLog.pdb
error APPX1101: I:\Documents\Projekty\stupid things\MetroLogDepApp\packages\MetroLog.1.0.1\lib\portable-net45+wp8+win8+wpa81\MetroLog.pdb

error APPX1101: Payload contains two or more files with the same destination path 'MetroLog.pdb'. Source files:
error APPX1101: I:\Documents\Projekty\stupid things\MetroLogDepApp\packages\MetroLog.1.0.1\lib\wpa81\MetroLog.pdb
error APPX1101: I:\Documents\Projekty\stupid things\MetroLogDepApp\packages\MetroLog.1.0.1\lib\portable-net45+wp8+win8+wpa81\MetroLog.pdb

Attached is an empty Win81/WPA81 project with a portable library targeted to Win81/WPA81. This combination triggers the error for me.
MetroLogDepApp.zip
I could, supposedly, use different target for the universal Win81 project, but is it reasonable?
This issue started appearing since 1.0.0.

Can you try converting your packages.config to project.json as detailed in this blog post:
https://oren.codes/2016/02/08/project-json-all-the-things/
It should solve your issues as project.json handles transitive references significantly better.

I tried it on that simple project and it indeed did solve the issue! Thanks for this, I'll try to convert it in our big project and we'll see how that goes.

Aaand indeed it helped! Thank you again, I'm closing it, I hope you won't mind :)


Convert string and char_buf matches to hash look ups

Currently string, char buf, file, fd, and path matchers (equal, prefix, postfix, plus the 'not' equivalents for file, fd and path) are all written in BPF using loops to inspect (potentially) every character in the strings to be matched.
This series (see commits for details) converts all these matchers to hash map look ups, using a set of 6 hash maps of varying sizes for equal matches, and LPM TRIE maps for prefix and postfix look ups. In addition, the 'not' equivalents are implemented for string and char buf.
The default string lengths for the matches has been set at 128 bytes, but could be increased to an arbitrary size for equals, and up to 256 bytes for prefix and postfix. For small programs (<5.3), the postfix limit (which is bounded by a byte-by-byte reverse copy) has been raised from 40 chars to 96 chars. For large programs, the postfix limit is set at 128 chars (raised from 50 chars).

In any case, I would suggest that the keep the go structures simpler than what we do in bpf, and then deal with the exact structure of the bpf maps when we load the data.

The problem with keeping the go structures simpler is that we need to know the array indices of the hash maps at the point that we build the kernel selectors. The easiest way to know these is to make lists of hash maps that mirror those in BPF. That said, some level of harmonisation is possible, which I shall do.

One thing I'm not so sure about is whether it is worth the complexity of having multiple maps instead of one. As far as I understand the motivation here is to save space.

, and it would be nice to justify the code complexity of having multiple maps with some numbers.
The motivation is to save CPU cycles in the look up code. If we only have one hash map with a key size of (say) 128 bytes, then every string we try to look up will require hashing 128 bytes, regardless if many of them are 0x00. Using several maps with different size keys means we only hash an amount of bytes that is closer to the string length we wish to look up.

During offline sync we discussed quickly why the different maps, and to me it is fine. Maybe it is complex from readability, but from the performance view even if we don't have numbers, but the fact that 1. these selectors where not working properly and now they are being fixed, and 2. file name and string matching is the most basic and easy use case. Users may watch different paths then having better benchmark results on top is always good.

(1) and (2) make sense to me, but they could also be implemented with a single map for all sizes. Having different maps for different sizes is a performance optimization (for CPU cycles as you mention above), and, given the code complexity it introduces, I think it's reasonable to ask: "is it worth it?" How much time do we save compared to when we use a single map for all sizes.

(1) and (2) make sense to me, but they could also be implemented with a single map for all sizes. Having different maps for different sizes is a performance optimization (for CPU cycles as you mention above), and, given the code complexity it introduces, I think it's reasonable to ask: is it worth it? How much time do we save compared to when we use a single map for all sizes?

Jhash cost is a little over a CPU cycle per length of data to hash in bytes, but we could say that is equal; i.e. hashing 128 bytes costs 128 CPU cycles (see https://fosdem.org/2023/schedule/event/bpf_hashing/ for details/confirmation). The existing byte-by-byte comparison has a cost of CPU cycles that similarly grows with the length of the data to compare.
While fixing (1) and (2) above, we also don't want to incur extra CPU cost if we can avoid it. In this situation, we want to give the users the ability to specify fairly long strings/filenames to match on – say ~128 bytes. If we take the simplistic approach of a single map, then all look ups will require hashing 128 bytes, with a cost of ~128 CPU cycles. By dividing the strings into different maps, we reduce this cost to approximately the size of the key. Therefore, assuming many look up strings are less than 24 byte in length, many look ups would cost 25 CPU cycles compared with 128 CPU cycles. We could assume that the next biggest category would be slightly longer strings, with these costing 49 CPU cycles compared to 128 CPU cycles.
Compared to these gains, the code isn't really that complex. Rather than putting all the strings in the same map, they are simply divided into six maps based on their length; and then at look up, the appropriate map is chosen based on the length of the look up string. An advantage of this approach is that we could increase the maximum string length to, say, 1024 bytes with no additional CPU cost – there may be a small memory cost, however.
Given that Tetragon hooks kernel functions and performs comparisons during many of the hooks, we always cost a system CPU time. Looking at Sysmon (Windows) as an example of a tool with millions of installs, many of the use cases feature string and filename matching. IMO it's not a case of "Is it worth it?" but more a case of "How could we justify not doing it?"

I've taken some measurements using 50,000,000 calls to a kprobe:

The difference between no hash and the hash based on 6 maps is 60.2ns/kprobe.
The difference between no hash and the hash based on a single map is 127.6ns/kprobe.
The difference between the hash based on 6 maps and the hash based on a single map is 67.4ns/kprobe.
Objectively, the hash for a single map is more than twice the cost of the hash for 6 maps. This is admittedly very small compared to the total cost of the kprobe (2.92us), and represents an increase of 2.3%.

I'm getting following error for spec below, which I assume should work now
time="2023-09-06T13:17:54Z" level=fatal msg="Failed to start tetragon" error="sensor gkp-sensor-1 from collection sys-symlink-passwd failed to load: failed prog bpf/objs/bpf_multi_kprobe_v61.o kern_version 393730 loadInstance: failed to insert string_maps_0_8: update: key too big for map: argument list too long"
it seems there's double values iteration, first in writeMatchValues and then in writeMatchStrings,
but I still don't get fully the mapid/map_idx correlation.. so I might be off ;-)
apiVersion: cilium.io/v1alpha1
kind: TracingPolicy
metadata:
  name: "sys-symlink-passwd"
spec:
  kprobes:
  - call: "sys_symlinkat"
    syscall: true
    args:
    - index: 0
      type: "string"
    - index: 1
      type: "int"
    - index: 2
      type: "string"
    selectors:
    - matchArgs:
      - index: 0
        operator: "Equal"
        values:
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"
        - "AAAAAAAAAAAA"

I'm getting following error for spec below, which I assume should work now
time="2023-09-06T13:17:54Z" level=fatal msg="Failed to start tetragon" error="sensor gkp-sensor-1 from collection sys-symlink-passwd failed to load: failed prog bpf/objs/bpf_multi_kprobe_v61.o kern_version 393730 loadInstance: failed to insert string_maps_0_8: update: key too big for map: argument list too long"
it seems there's double values iteration, first in writeMatchValues and then in writeMatchStrings, but I still don't get fully the mapid/map_idx correlation.. so I might be off ;-)

That's odd. I'll take a look. Thanks for trying it out.

it seems there's double values iteration, first in writeMatchValues and then in writeMatchStrings, but I still don't get fully the mapid/map_idx correlation.. so I might be off ;-)

Yep that's what it was – I fixed it in the last push. When I simplified the code I moved writeMatchStrings into writeMatchValues as I thought it made more sense there, but this introduced the double iteration. All looks good now.
Regarding mapid/map_idx: consider just the smallest key'd map (key size is 25 bytes). A single kprobe could have more than one string Equal match section – maybe for different parameters or different actions. Therefore there could be 2 or more hash tables of strings, one for each match section. Each of these is placed inside an array_of_hash map at indices 0, 1, 2... These indices are then stored in the kernel selectors, so each match knows which hash table to use.
But of course there are six key sizes, so therefore six array_of_hash maps, and therefore the above process six times. In the selector we store the index for the relevant hash map in the first array_of_hash, then the index for the relevant hash map in the second array_of_hash, and so on for all six. In BPF, we take the input string, work out which key size it would fall into, get the index from the selector for that hash map, and look it up in the relevant array_of_hash. Then we can pad our string to that key size and look up the string in that hash map.
HTH

I'd like to see more tests given the size of new code (also for prefix/postfix), but I guess that can be added later, anyway it looks good to me now

Thanks for the approval. My thought about tests was that this isn't new functionality, so while more tests might be a good idea, we were already okay with the testing of the existing implementation (which happened to have bugs that I fixed a little while back!).

I'd like to see more tests given the size of new code (also for prefix/postfix), but I guess that can be added later, anyway it looks good to me now

Thanks for the approval. My thought about tests was that this isn't new functionality, so while more tests might be a good idea, we were already okay with the testing of the existing implementation (which happened to have bugs that I fixed a little while back!).

yep understandable, it would be nice though to add some tests on the limits, like the one Jiri proposed https://github.com/cilium/tetragon/pull/1408#issuecomment-1708358266. But as he said, we can add them later.

yep understandable, it would be nice though to add some tests on the limits, like the one Jiri proposed #1408 (comment). But as he said, we can add them later.

I added tests to the first commit for the 6 map string look up, all focused on the limits. Prefix and postfix could potentially benefit from tests at the max string length, but I don't feel they are urgent. The fact that tests already exist for them (from the previous implementation) satisfies me.


fix minor bash issues in get_info.sh

[Current coverage][cc-pull] is 90.09%

Merging [#107][cc-pull] into [master][cc-base-branch] will not change coverage

@@             master       #107   diff @@
==========================================
  Files            22         22
  Lines          2513       2513
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
  Hits           2264       2264
  Misses          249        249
  Partials          0          0

Powered by Codecov. Last updated by [75b3d7c...61f779b][cc-compare]
[cc-base-branch]: https://codecov.io/gh/CyberReboot/vent/branch/master?src=pr
[cc-compare]: https://codecov.io/gh/CyberReboot/vent/compare/75b3d7c20cdfff2908eda2e486f6e5425cc33fbe...61f779b48c08275d29ad0f715a51f615f40bd56d?src=pr
[cc-pull]: https://codecov.io/gh/CyberReboot/vent/pull/107?src=pr


Xiaomi Walkingpad

Device ID: 3153…
Model info: ksmb.walkingpad.v1
Address: <IP_ADDRESS>
Token: <… token …> via auto-token
Support: At least generic

Type info: miio
Capabilities:

Firmware version: 1.6.7
Hardware version: MTK7697
MCU firmware version: 0057

WiFi:  (00:00:00:00:00:00) RSSI: 0
WiFi firmware version:<PHONE_NUMBER>1057

Remote access (Mi Home App): Maybe

Accidentally discovered that you can get speed with
$ miio protocol call <IP_ADDRESS> get_prop '["speed"]' --token <token>
INFO  Got result:
[
  4
]

and set speed via
miio protocol call <ip> set_speed '[3]' --token <token>

Another supported property seems to be mode: get_prop ["mode"] and set_mode [1]. 0 is automatic, 1 is manual, 2 is standby.

Found that you can get all props:
$ miio protocol call <ip> get_prop '["all"]' --token <token>
[
  "mode:1",
  "time:171",
  "sp:4.0",
  "dist:180",
  "cal:11630",
  "step:271"
]

Found that you can get all props:
$ miio protocol call <ip> get_prop '["all"]' --token <token>
[
  "mode:1",
  "time:171",
  "sp:4.0",
  "dist:180",
  "cal:11630",
  "step:271"
]


fix for edit contact profile picture

Issue #:
https://github.com/bcgov/met-public/issues/991
Description of changes:
Changes made to check if edited contact has the image file name in the form then use it else use fetched data.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the met-public license (Apache 2.0).

Codecov Report

Merging #1109 (d399026) into main (8077c81) will increase coverage by 5.77%.
The diff coverage is 50.00%.

@@            Coverage Diff             @@
##             main    #1109      +/-   ##
==========================================
+ Coverage   68.63%   74.40%   +5.77%
==========================================
  Files         153      244      +91
  Lines        3443     6983    +3540
  Branches      475      477       +2
==========================================
+ Hits         2363     5196    +2833
- Misses       1004     1711     +707
  Partials       76       76

Flag
Coverage Δ

metweb
68.62% <50.00%> (-0.02%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...agementWidgets/WhoIsListening/AddContactDrawer.tsx
53.01% <50.00%> (-0.08%)
:arrow_down:

met-api/src/met_api/models/widget_documents.py
97.14% <0.00%> (ø)

met-api/src/met_api/services/contact_service.py
57.14% <0.00%> (ø)

met-api/src/met_api/schemas/engagement_survey.py
100.00% <0.00%> (ø)

met-api/src/met_api/models/pagination_options.py
100.00% <0.00%> (ø)

met-api/src/met_api/models/user.py
93.10% <0.00%> (ø)

met-api/src/met_api/constants/widget.py
100.00% <0.00%> (ø)

met-api/src/met_api/models/contact.py
72.09% <0.00%> (ø)

met-api/src/met_api/utils/roles.py
100.00% <0.00%> (ø)

met-api/src/met_api/__init__.py
97.43% <0.00%> (ø)

... and 82 more


Update bezier with new implementation

Pulled refactor from @gre here https://github.com/facebook/react-native/commit/b5985cf6908365e28bff1f3dd15c44d7586a6433 and moved it over for issue #20

:+1:


Fix typos and improve clarity in scripts and error handling

`This PR addresses several issues related to typos and minor improvements in various scripts and codebases. Key changes include:
Corrected a typo in a variable name to ensure proper functionality.
Fixed missing spaces in git commit commands for better readability and execution.
Resolved an extra bracket issue in the authors' error message, improving its clarity.
Updated the spelling in the project initialization step for accuracy.
Replaced "console.log" with "console.error" in error handling to better reflect error contexts.
These changes aim to enhance the codebase's readability, robustness, and functionality.`

Thanks for your contribution @Miska05 but we prefer to have more substantial first-time contributions, please feel free to look through our good first issues to find something that may interest you


Add options

ipv6, to get v6 only listening
host, to specify where to listen
initial, run exec on startup
timeout, to kill -TERM exec after these ms
dir can be specified in config file

handles multi-glob without creating multiple watchers,
so that initial only fires once
handles port 0 better
improved watchGlob error handling, so exec errors
get logged cleanly without killing server
use hostlocal to develop hostlocal: npm run dev
moved away from .npmignore, to putting explicit
directories in package.json/files.

Codecov Report
Attention: Patch coverage is 97.97980% with 2 lines in your changes missing coverage. Please review.

Project coverage is 98.41%. Comparing base (8176460) to head (1a6e080).

Files with missing lines
Patch %
Lines

src/index.ts
94.11%
2 Missing :warning:

Additional details and impacted files
@@           Coverage Diff           @@
##             main      #16   +/-   ##
=======================================
  Coverage   98.40%   98.41%
=======================================
  Files          11       11
  Lines         943     1010   +67
  Branches       87       95    +8
=======================================
+ Hits          928      994   +66
- Misses         14       15    +1
  Partials        1        1

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Method signature info overflows the box

Steps to Reproduce:

Type vscode.commands.executeCommand()
On the parenthesis, the info overflows 🐛

@alexdima or @mjbvz did you change anything here?

@alexdima or @mjbvz did you change anything here?

AFAIK I didn't touch the parameter hints, I just did some refactorings in the modes content hover.

AFAIK I didn't touch the parameter hints, I just did some refactorings in the modes content hover.

Seeing the same thing with contents overflowing the hovers.
I wonder if its related to https://github.com/microsoft/vscode/issues/111572

Seeing the same thing with contents overflowing the hovers.
I wonder if its related to https://github.com/microsoft/vscode/issues/111572

@jrieken Possibly d66e65fcdd90644685f4afa4ed369dad0e189a95 but that was merged back on Dec 8th. It just added an extra call to scanDomNode after the markdown finishes rendering

@jrieken Possibly d66e65fcdd90644685f4afa4ed369dad0e189a95 but that was merged back on Dec 8th. It just added an extra call to scanDomNode after the markdown finishes rendering

Oh actually, @alexdima I think it's cbea242dcbb3709a8cecc59f442d897515bba502 which changed the class name but didn't update this location:
https://github.com/microsoft/vscode/blob/cbea242dcbb3709a8cecc59f442d897515bba502/src/vs/editor/contrib/parameterHints/parameterHintsWidget.ts#L372
Checking now

Oh actually, @alexdima I think it's cbea242dcbb3709a8cecc59f442d897515bba502 which changed the class name but didn't update this location:
https://github.com/microsoft/vscode/blob/cbea242dcbb3709a8cecc59f442d897515bba502/src/vs/editor/contrib/parameterHints/parameterHintsWidget.ts#L372
Checking now

Yes that was it. Just pushed a change that updates the class name, which fixes the issue for me

Yes that was it. Just pushed a change that updates the class name, which fixes the issue for me

Oh no, very sorry about that!

Oh no, very sorry about that!


Implement Tests to Read Balance and Block Details to Verify Proper Transaction Storage

Why
Ensuring that all transactions are properly stored and reflected in the balance and block details is crucial for the integrity and reliability of the Stratus system. These tests will help detect any discrepancies in transaction storage, which can indicate underlying issues in the consensus mechanism or storage backend.
How

Read Balance Test:

Implement a test that reads the balance of accounts after a series of transactions.
Verify that the balances reflect all executed transactions accurately.

Read Block Details Test:

Implement a test to read block details, including transactions, from the blockchain.
Verify that all transactions are present and correctly stored within the blocks.

Test Suite Integration:

Integrate these tests into the existing test suite to run automatically.
Ensure the tests cover various scenarios, including high transaction volumes and edge cases.

We've shelved raft for now, closing.


Update CONTRIBUTING.md

Added more details, formatted links.
I also re-formatted the table as the previous version was (maybe auto-generated and) a pain to edit. Sorry if this throws off your editor.

r? @brendandahl

There goes my pretty ascii table layout. ;)


pls help!!cannot fit 'int' into an index-sized integer

pls help!!!

my mistake，the file name of tokenizer not 100% mached


what is AutoModelForCausalLMWithValueHead?

trl use AutoModelForCausalLMWithValueHead，which is base_model(eg: GPT2LMHeadModel) + fc layer，but I can't understand why need a fc head layer?

In PPO optimization a value estimation for each action (=generated token) is required. The value head (which is just an additional fc layer) makes that estimation.

ok, thanks

Hi @lvwerra , thanks for the implementations! A n00b question: in my understanding, seems the goal of value head is to estimate the KL divergence between policy and reference model so that they do not differ too much. And in PPO training, the entire reference network is not updated. My question is:
I am wondering, when initializing reference network with additional value head, is this value head randomly initialized? Also I think probably the value head is updated in training, otherwise the output won't make sense due to random initialization. If so, wondering what's the label used to train it? Very curious to learn some details behind it.
I think without value head, we can use some likelihood to estimate pi_RL(y|x) and pi_SFT(y|x), but seems with it we can just use two scalars from value head to represent two probabilities. Wanted to learn if this understand is correct. Thank you so much!


Add the expected signature for missing term member problems.

This is for MissingTermMember and NewAbstractMember. It can help identify problems in the presence of overloads.

Ping @bishabosha


update AerBackend docs and class highlighting

I've updated the documentation of AerBackend and added some highlighting for the different python classes in the index pages.
Let me know if you think more should be added. It might be nice to explain more about how AerBackend relates to the IBM simulator and maybe show how to define a NoiseModel or custom architecture.

Use of IBMQ is already deprecated (in favour of QiskitRuntimeService) and will soon be removed -- at that point we will have to rewrite some of the authentication machinery and update these docs. But we'll cross that bridge when we come to it.