Remove six as a dependency in core

Came in a discussion with @annatisch , and we feel it's probably time, that it wouldn't break anyone, and thee few if any would have done wrong things in the first place.

API change check
API changes are not detected in this pull request.

/azp run python - identity - ci

/azp run python - tables - ci


a vs _a

In the docs, mv uses _a but as matrix uses a. Are these meaningfully different parameters or could they be made the same?

The underscore means that it is not checked. This reflects the fact that it's in a PyTree, and different nodes in the PyTree need not have the exact same sizes.


fixfilter impala generated path

Proposed changes
meger master: https://github.com/apache/doris/pull/28786

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall


Decrease change percentages has negative sign

In the change percentage, when there's a decrease the percentage change has a negative sign. This looks like:
This shows a decrease of -90.9% compared to the previous 7 days.
(Source: Simple summary - Barnsley on 21 December 2021)
This is problematic as the decrease is not negative. I suggest the behaviour should remove the negative sign from the change percentage, like this:
This shows a decrease of 90.9% compared to the previous 7 days.

I think this can be resolved using the abs nunjucks filter.


Problems with documentation TOC

Several .. toctree:: commands in index.rst make following problems:

On page security.html Previous topic is Contrib Modules instead of PoolManager
Page Collections didn't include in "Next topic chain".

security included in main toctree twice (begin of index.rst) but exceptions not once.
Also there is one build warning: "urllib3\docs\security.rst:117: WARNING: duplicate label pyopenssl, other instance in urllib3\docs\contrib.rst".

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/23305952-problems-with-documentation-toc?utm_campaign=plugin&utm_content=tracker%2F192525&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F192525&utm_medium=issues&utm_source=github).

@Disassem Any fixes/improvements to the docs are super-appreciated. Would you like to do a PR? :)

Just now I don't know how to fix it. I'll think about this :)


Bug: Student files not available to autotesting module.

I have run the same test cases in MarkUs on submissions (without changing the submissions) and sometimes received an error from the automarker that no student files are present for marking and other times received the expected mark. This could be an automarker bug, but I'm guessing it is a MarkUs bug that results from not syncing (or not waiting for) files passed to the marking system to be written to disk before marking commences.
You are using MarkUs version: 1.5.0.DEV

Ruby version: 2.2.5

Rails version: 4.2.4```

This has been fixed!


Implement routing configuration for Backstage server endpoint

Changes made:

Autodetect if the current cluster is OpenShift and creates a route when Backstage CR is reconciled.
Changed backstage deployment and service name to "backstage-"+ to allow the users to deploy multiple backstage CRs in the same namespace.
Changed the default value of own-runtien to true.
Note #1: supporting host and tls in the route via Backstage CR is not in the scope of this PR. Such work shall be done later as a separate PR after the data model has been finalized.
Note #2: unit test with isOpenShift=true is not included as OpenShift specific objects (routes, projects etc) are not currently supported by the envTest tool (see operator-framework/operator-sdk#4434).

Which issue(s) does this PR fix or relate to

Fixes #53

PR acceptance criteria

[ X] Tests
[ ] Documentation

How to test changes / Special notes to the reviewer

Deploy the operator  on OpenShift and create the postgres secret (make sure POSTGRES_HOST is set to "backstage-psql-", and create backstage CR.
Check that a route has been created:
oc get route
NAME            HOST/PORT                                                        PATH   SERVICES        PORT           TERMINATION     WILDCARD
backstage-bs1   backstage-bs1-backstage.apps.cluster-s4l2f.dynamic.opentlc.com   /      backstage-bs1   http-backend   edge/Redirect   None
Now check if you can open the page in your browser:
https://

Closed this PR due to fork visibility issue and resubmitted: https://github.com/janus-idp/operator/pull/65


Please support passing in a youtube url!

Since spotdl actually downloads from youtube (or at least it used to), a youtube url should be the most reliable way to use this program.
Using search is very unreliable. I've just passed in 5 searches, spelt correctly for songs that spotify has and it only got 2 right. If i could have passed in the youtube urls, I would have been able to get all 5 songs.
The old spotdl allows this - I used to paste a bunch of youtube urls into a file and run spotdl -l filename.txt and let it run. That would be great if you could replicate that
Thanks!

Please follow our issue templates when reporting bugs or requesting features.
The devs put in hours of effort, and you should show enough respect to put in a few minutes & follow the template.
Following the template also provides us all the information we need, without having to ask for it.

Please follow our issue templates when reporting bugs or requesting features.
The devs put in hours of effort, and you should show enough respect to put in a few minutes & follow the template.
Following the template also provides us all the information we need, without having to ask for it.

Please reformat and open a new issue.

Please reformat and open a new issue.


Error displaying the dashboard

After logging in to CodeTime on codetime.dev with GitHub, I am seeing the following error in the route /dashboard.

Edit:
The bug only appears after data was uploaded to CodeTime via the extension.

Closed, because the issue was made in the wrong repository.


test_with_modified_rulebase_off_boundary fails in modify_rulebase_and_tst

When running plugin version <IP_ADDRESS> in server built from tip of main, test_with_modified_rulebase_off_boundary fails.
This failure occurs as a non-AssertionError exception being raised during the test, which results in the failure being logged as an error in the test.
The same failure occurs in my PR #140, so presumably this has not been fixed yet.
The test is running in an Ubuntu 20.04 container, using postgres.
Here's some of the test output. I've put a more complete log here.
======================================================================
ERROR: test_with_modified_rulebase_off_boundary (irods.test.test_prep_genquery_iterator.Test_Genquery_Iterator)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/lib/irods/scripts/irods/test/test_prep_genquery_iterator.py", line 648, in test_with_modified_rulebase_off_boundary
    self.boundary_with_size(512 + 7 * 8 * 9, self.modify_rulebase_and_tst, pre_start = True)
  File "/var/lib/irods/scripts/irods/test/test_prep_genquery_iterator.py", line 680, in boundary_with_size
    func(N)
  File "/var/lib/irods/scripts/irods/test/test_prep_genquery_iterator.py", line 810, in modify_rulebase_and_tst
    self.assertNotEqual( out_tuple[0][0], out_tuple[1][0]) # keys not a match ("actual" != "predicted")
IndexError: list index out of range

I believe this is fixed by changes made in #171. We observed this test failure in the run up to <IP_ADDRESS> and the tests are now passing.


runtime: Windows profilem may profile Ms after unminit

profileLoop partially synchronizes with unminit via mp.threadLock, but it drops threadLock prior to _SuspendThread and profilem.
Thus profiling may race with dropm, resulting in profilem running on an "extra" M which is no longer in use by Go and whose previous thread is running C code.
In theory, this should result in gFromSP returning nil (as logically there should not be a G at all since this isn't a Go thread), which may make sigprof crash, as it assumes gp != nil. However, in practice the SP likely still falls in the mp.g0 stack range and things end up not crashing, though with a non-meaningful profile sample.
I've been able to easily detect this condition in practice with a C thread repeatedly calling a Go function and profilem checking if mp.thread is still set.
cc @cherrymui

It looks like preemptM has a similar issue. It synchronizes with "external code" using mp.preemptExtLock, which is set by cgocall, but is not set by dropm/unminit. Like profileLoop, preemptM does not fully synchronize with unminit via mp.threadLock. As a result, I believe that preemptM could attempt to preempt a C thread after unminit, which may call ExitProcess.
cc @aclements

I see a few different options here:

Hold mp.threadLock until _SuspendThread completes (i.e., after _GetThreadContext), thus unminit can't run until we have stopped the thread.

I think this is the simplest approach, but does introduce possibility of contention on mp.threadLock. A preemptM call may have to wait a long time for profileLoop to unlock before it can continue.
On the other hand, maybe that is OK. In the standard case, preemptM is going to suspend the thread anyways, so in theory waiting for profileLoop too only makes preemptM take 2x as long.

Same as (1), but add an atomic flag that profile suspend is in progress. preemptM immediately fails preemption if this flag is set.
Clear mp.g0.stack on unminit and skip sigprof/preempt if gFromSP can't match up with a G.

I'm not a big fan of this because it still allows racing with needm picking up the M again and setting mp.g0.stack, which may not even be assigning the M to the same thread that we are profiling.

For preemptM, we can set mp.preemptExtLock in unminit, which will make it skip uninitialized Ms as "external code".

For profilem, do nothing? We get nonsense profiling samples, and could crash sigprof with a nil G, but we could skip profiling entirely on a nil G.

I think 4 is a good idea, for preemption. Not sure about profiling, maybe we could do 3. Maybe we put setting/unsetting stack bounds in the same lock. Or we could re-check mp.thread after the thread is suspended? (is there any ABA problem?)
Not sure about the implication of suspending thread with the lock held. I haven't thought it carefully, but it sounds a bit dangerous (for deadlock). Maybe it is fine.

2022-09-05T08:08:24-af7f417/windows-amd64-longtest
Is this exactly this issue? It looks really similar. A nil pointer dereference in profilem.

I believe that is #54885.


test(locales): Add locales test

As mentioned in #478.

Coverage remained the same at 34.504% when pulling 93a0a53d60c5ad225010ae8fa13370dd0e5b6c52 on chinesedfan:test_locales into 3714dc5ec3ad6fdf6b129c03078e1b849d546d12 on gitpoint:master.

Coverage remained the same at 34.504% when pulling 93b5a25638ce94c2ca010495e9c27e01ab72b600 on chinesedfan:test_locales into 3714dc5ec3ad6fdf6b129c03078e1b849d546d12 on gitpoint:master.


Automate setting of backstage.global.clusterRouterBase

Can we replace the manual setting of backstage.global.clusterRouterBase with a lookup function?
oc get ingress.config.openshift.io/cluster -oyaml | yq '.spec.domain'

The lookup should be like:
{{- $cluster := lookup "config.openshift.io/v1" "Ingress" ".Values.global.rhsso.namespace" "cluster" }}
{{- $domain := "UNKNOWN" }}
{{- if hasKey $cluster "spec" }}
  {{- if hasKey $cluster.spec "domain" }}
    {{- $domain = $cluster.spec.domain}}
  {{- end }}
{{- end }}

@masayag

Adding a reference to the usage of the property:
https://github.com/janus-idp/helm-backstage/blob/a20e258fa478970baf4a1db2ac1e6bae1e3332de/charts/backstage/templates/_helpers.tpl#L7

I missed the fact that this is used in a dependant chart, and helm does not provide a way to override a template function.
so, assuming that Backstage deployers are already familiar with using this global flag, we can probably just accept this as is (or maybe create ab equivalent issue in the upstream project, so they can solve it instead)

Tracked new issue https://github.com/janus-idp/helm-backstage/issues/167 to dependant chart


Corobo CI Failing due to old pytest version

The corobo build is failing due to a old version to pytest.
Plugin 'xdist' could not be loaded: (pytest 3.6.4 (/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages), Requirement.parse('pytest>=4.4.0'))!
Difficulty : newcomer

Should be closed? @sladyn98

@abhishalya does the build succeed?

Yep, https://travis-ci.org/coala/corobo/builds


Update test now that code is fixed

processing of escaped keys was fixed for get_json_object in https://github.com/NVIDIA/spark-rapids-jni/pull/2410
This updates the test to not xfail any more. I am not sure why I put https://github.com/NVIDIA/spark-rapids/issues/11386 as the issues that is making the one test not work because it is not correct.

build

build


BUG: missing check on minkowski w kwarg

fixes #8142

Thanks @apbard. Could you add a regression test?

TravisCI test failure is unrelated, that's the updateifcopy deprecation.

Could you add a regression test?

I forgot to push it...added it now.

LGTM, merged. Thanks @apbard


RELENG-5962 implement gcloud backend

Codecov Report

Merging #113 (caa9abb) into main (2594c5f) will increase coverage by 0.08%.
The diff coverage is 27.27%.

@@            Coverage Diff             @@
##             main     #113      +/-   ##
==========================================
+ Coverage   70.10%   70.18%   +0.08%
==========================================
  Files          15       15
  Lines         796      795       -1
==========================================
  Hits          558      558
+ Misses        238      237       -1

Impacted Files
Coverage Δ

..._manager/vm_creation/openstack/OpenstackManager.py
29.52% <ø> (+0.46%)
:arrow_up:

srcs/runners_manager/vm_creation/CloudManager.py
52.50% <27.27%> (-9.57%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2594c5f...caa9abb. Read the comment docs.

I think we should find a solution too keep the tags in the runners names


Document the DisplayServer class

This makes the DisplayServer class 100% documented.
This also adds a link to the Command line tutorial on pages that reference command line arguments, as the page covers some general usage tips for CLI arguments (especially on macOS).

Thank you, merged! 🎉


Feature Request: Evaluation Notifications

in short:
As a lorri user I would like to be able to be notified whenever an evaluation has finished and know whether it was successful or not.
current scenario:
I am working on a Haskell project and add a dependency to my cabal file. In order to know if this was successful and when I can start using it I have to open my my (otherwise hidden because mostly uninteresting) tmux pane with the lorri process.
desired scenario
I am working on a Haskell project and add a dependency to my cabal file. Everything is built and re-evaluated successfully and lorri triggers a notification which I can see through my dbus notification app.

Some additional thoughts

I understand lorri will eventually be running as a daemon in the background. In this case support for a notification like this might be even more desirable.
I think #64 is related to this and it would in fact be possible to use this generic mechanism to trigger notifications using something like notify-end yet I believe that being informed about the state of the evaluation is important enough to be a feature in its own right.
Instead of only notifying about successful or unsuccessful evaluation completion it might also be useful to include evaluation start. Ideally this would be configurable on the command line.
https://github.com/hoodie/notify-rust

Looking forward to hear your thoughts on this. Thanks

We could add a --notify flag to the tool that watches for changes.
Would you expect to be notified about every evaluation on every project the daemon watches, or just for some subset of projects?

@Profpatsch Oh that's a good question. I think I would say every project because everything else would probably be inconvenient and unexpected. Plus evaluations should only occur if I actually do change something in the project and then i would also want to be informed about it.

It seems this feature has some intersection with #64 to emit events


:sparkles: Customizable SoloModel cache prefix

From https://github.com/open-formulieren/open-forms/pull/592#discussion_r806022212

Codecov Report

Merging #35 (734b164) into master (0975973) will decrease coverage by 0.24%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master      #35      +/-   ##
==========================================
- Coverage   92.91%   92.66%   -0.25%
==========================================
  Files          22       22
  Lines         536      518      -18
==========================================
- Hits          498      480      -18
  Misses         38       38

Impacted Files
Coverage Δ

mozilla_django_oidc_db/models.py
75.29% <100.00%> (+0.90%)
:arrow_up:

tests/test_backend.py
100.00% <0.00%> (ø)

tests/test_settings.py
100.00% <0.00%> (ø)

tests/test_admin_form.py
100.00% <0.00%> (ø)

tests/test_middleware.py
100.00% <0.00%> (ø)

mozilla_django_oidc_db/mixins.py
100.00% <0.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0975973...734b164. Read the comment docs.


Memory leak while allowing player to wait in loading screen on UE 4.25.4

Hey Truong, these settings will cause a Windows 64-bit build to leak memory if you stay on the loading screen:

Wait For Manual Stop = true
Minimum Loading Screen Display Time = -1
Auto Complete When Loading Completes = false
Movies are Skippable = false
Allow in Early Start = false
Allow Engine Tick = false

I did a memreport both before and after the loading screen and noticed this difference:

Hi @terishrimp, thank you for reporting bug, I'll investigate this issue.

@terishrimp, you shouldn't worry about the memory leak in Async Loading Screen. All the pointers in the plugin code are shared pointers, which will be garbage collected in the end anyway. There is no manual memory allocation in the plugin source code, I think it may take an amount of time for GC to collect all the Slate pointers. However, if there is a memory leak in the Slate module or MoviePlayer module, it's out of my ability.

Hey @truong-bui, fair enough. I have tried allowed the level proceeding the loading screen ample time for GC to see if the memory decreases but it doesn't. Thanks for looking into it! I'll go ahead and do further research myself.


DOC: Fix 'missing-bits' document ref in the contributor docs

I think this was missed in the restructuring that was done in https://github.com/scipy/scipy/pull/15947.

It looks like this change fixes the problem.  The fixed link is the last item in the Editing SciPy section, and it links to Code and Documentation Style Guide - The Missing Bits.

Hmm, it seems to be working already in the devdocs if you click through from here, no? https://scipy.github.io/devdocs/dev/contributor/contributor_toc.html

Ah, you're right.  I guess I was looking at https://docs.scipy.org/doc/scipy/dev/contributor/contributor_toc.html, where the link is missing, but that refers to SciPy 1.8.1.  (Of course, I should have noticed this by building the docs locally, but my recent attempts to build the docs locally failed, and I haven't yet looked into why.)


HTTP Package delete action message is mandatory

Description:
In HTTP package delete action require you to pass a message parameter, but as for HTTP spec it may or may not have a body. So shall we change the action signature similar to GET.
Affected Versions:
0.982.0

Yes, we should make the message as an optional to DELETE action.
As per the specification https://tools.ietf.org/html/rfc7231#section-4.3.5
A payload within a DELETE request message has no defined semantics;    sending a payload body on a DELETE request might cause some existing    implementations to reject the request.


Add NOTICE and LICENSE

Type

Documentation

Related JIRA Issue(s)
ENSWBSITES-504
Description
Add the NOTICE, LICENSE and README for this repository. The README is still a work in progress.

The README will have a separate PR and @ens-ap5 is working on it. This PR is now only for the LICENSE and NOTICE.

There is a README.md now in dev (looks super placeholder-y; don't know why it wasn't written down in a PR). It probably means that this PR should not include README.md in its diff.

@azangru Removed README.md in this PR branch


updated monolingual (1.7.3)

Thank you for the contribution. It was merged directly as 3852d0a2450c8ab485014b434f1a24986ae70519 to keep commit history cleaner. Your contribution is still credited to you.


being able to set custom rspec load path

Subject of the issue

I'm looking at a legacy app that has a lot of ruby code in an api folder on the root, and a fair amount of custom code to load it to be accessible by the specs.  I'm wondering if there's a simple config option to allow it to be auto-loaded as if the code was in the app or lib directories?
I've been reading through the docs but not immediately seeing a way to do this https://relishapp.com/rspec/rspec-core/v/3-11/docs/configuration
Your environment

Ruby version: 2.7.3
rspec-core version: 3.10.2

Steps to reproduce

create an rspec ruby setup, but place the ruby code under test in a folder called api
Expected behavior

by specifying code is in an api folder by some config specs within spec/api would find code in the root api folder
Actual behavior

custom loading required to auto load code under test in api folder
I realise the response may be, just put your code in lib/api but I'm just wondering if there's a hidden config option to make allow us to refactor our legacy app without changing directory structure in the first instance
Many thanks in advance 🙏

Ah, could I do it with?
RSpec.configure do |c|
  c.default_path = 'api'
end

RSpec doesn't do any autoloading, thats down to other libraries (as in Rails, where its Zeitwerk et al) or in built Ruby semantics (such as the autoload directive). You can tweak which folders are available in this fashion by tweaking the ruby options to have ./api on your load path (even programatically via $LOAD_PATH, which is what gems do in their gemspecs) but this is out of scope for an RSpec feature.

thanks @JonRowe so then this RSpec config that adds lib/ to directories is not controlling which directories are checked for ruby code when RSpec is running? https://github.com/rspec/rspec-core/issues/1983

The config option is to allow you do e.g. --require 'spec_helper' and have it mean spec/spec_helper.rb, like gemspecs we also add lib at that point as discussed, but there is no autoloading done, just a manipulation of the load path which you can do yourself without our help


Need help changing background to color

The background color is current set as an default white
It should be changed in the /app/globals.css file
The color should be changed to red.

Hey use-emmet, I need help!

Hey use-emmet, I need help!

Hey use-emmet, I need your help!

Hey use-emmet, I need your help!

Hey use-emmet, I need your help!

Hey use-emmet, I need your help!

Hey use-emmet, I need your help!

Hey use-emmet, I need your help!

Hey use-emmet, I need your help!

Hey use-emmet, I need your help!

Hey use-emmet, I need your help!


Version 2.0.0-alpha.25

Landed in https://github.com/metarhia/impress/commit/535f986e6281c3688bc13a569d8a407236435e2a and https://github.com/metarhia/impress/commit/9dfd72a4015e15aa4fa99ab124f4c4cc605d5b0c


Ability to sort icons in the collection by name.

Ascending/descending/none
Can be another option in collection menu. On each click of menu item the sort changes.

Hey @MrRobz  I would like to help on this one

Hello @MrRobz, I tried to implement this as part of hacktoberfest, could you please take a look at PR that I opened? https://github.com/Icon-Shelf/icon-shelf/pull/142


AttributeError: This FileResponse instance has no content attribute. Use streaming_content instead.

What happened?
AttributeError: This FileResponse instance has no `content` attribute. Use `streaming_content` instead.
I am getting this error while clicking on any media download link in django admin. I checked inside docker container and files are stored properly. I have used cookiecutter to start the project.
When i
What should've happened instead?
File should be downloaded.
Additional details

Host system configuration:

Version of cookiecutter CLI (get it with cookiecutter --version): Cookiecutter 1.7.3 from C:\Python310\lib\site-packages (Python 3.1)

OS name and version:
On Linux, run
lsb_release -a 2> /dev/null || cat /etc/redhat-release 2> /dev/null || cat /etc/*-release 2> /dev/null || cat /etc/issue 2> /dev/null

On MacOs, run
sw_vers

On Windows, via CMD, run
systeminfo | findstr /B /C:"OS Name" /C:"OS Version"

# OS Name:                   Microsoft Windows 10 Home Single Language
# OS Version:                10.0.19045 N/A Build 19045

Python version, run python3 -V: Python 3.12.3

Docker version (if using Docker), run docker --version: Docker version 26.1.4, build 5650f9b

docker compose version (if using Docker), run docker compose --version: Docker Compose version v2.27.1-desktop.1

...

Options selected and/or replay file:
On Linux and macOS: cat ${HOME}/.cookiecutter_replay/cookiecutter-django.json
(Please, take care to remove sensitive information)

Logs:

$ cookiecutter https://github.com/cookiecutter/cookiecutter-django
project_name [Project Name]: ...

Weird I never seen this problem

Having the same issue. Any updates?

Would be helpful if one of you could share the options they used to generate their project, as there is very little to go by right now. Help us help you 😄

Hi there,
the Django Debug Toolbar is responsible for the Attribute Error, deactivating all options, than reactivating solved the issue. Sorry for the inconvenience.


Revert "[BEAM-9615] Map user types to Schema reps."

Reverts apache/beam#12554
Broke every test build, because gogradle isn't awesome.

R: @lukecwik


[FR]  Multiple python packages in a single repository

What's the problem this feature will solve?
Prerequisites:

OS: Ubuntu 22.04
Python 3.11
setuptools version: 65.5.1
I'm targeting pypi

Hi I'm the creator of an open source program called GridCal. This consists of a Qt-based GUI and the calculation engine. Since the beginning, I've had the GUI and the engine under the same package for practical reasons. The structure below worked nicely and the packages for pypi were created correctly. This is because the setup.py file is at the same level as the package GirdCal
repository_folder
|
|_ src
    |_ GridCal
    |     |_ GUI
    |     |_ Core
    |     |_ IO
    |     |_ Simulations
    |     |_ __init__.py
    |
    |_ setup.py
    |_ upload_to_pypi.py

Recently, I've split the code in 2 packages (GridCal and GridCalEngine) to see if I can get a structure that works for development and for deployment to pypi.
the structure is the following:
repository_folder
|
|_ src
    |_ GridCal
    |     |_ GUI
    |     |_ __init__.py
    |     |_ setup.py
    |
    |_ GridCalEngine
    |     |_ Core
    |     |_ IO
    |     |_ Simulations
    |     |_ __init__.py
    |     |_ setup.py
    |
    |_ upload_to_pypi.py

This split structure works very well for development since the GUI can properly reference the engine, the generated packages are checked and uploaded to pypi, but when installing those packages, the installation fails because the setup.py file is not outside the package like in the previous example.
It would be very very useful to be able to tweak which setup.py file is being called and still produce installable packages with the expected structure. This is, with setup.py at the root of the .tar.gz package as the folder containing the python module
Describe the solution you'd like
I want to target any setup.py file (anywhere in the package structure) and produce pip-correct packages.
Alternative Solutions
No response
Additional context
No response
Code of Conduct

[X] I agree to follow the PSF Code of Conduct

Hi @abravalheri than a lot for the detailed answer!
My use case is that I want to distribute the calculation engine (with less dependencies) for users that only need to do scripting, and later if the install the GUI they can run the calculations visually. Hence I want to be able to deploy them as separated packages.
The GUI and the Calculation Engine need to be in the same development version, since I want to avoid frontend-backend versioning hell by design. This is, to develop both in the same repository as if they were the same package.
I know this is not what people do (separating in different package structures) but this is a very common use case, that at the moment is being forced into forced into different development structures.
So it would be nice to copy the setup (wherever it is on the structure) to the right place when building the tar.gz or whl file.

Hi @SanPen, in principle there is no problem in having multiple distributions inside of a single repository. The problem comes if you try to "reach out" of the directory that contains the config file.
In your case, the setup.py file seems to try to reach out for its parent directory. If you modify that, it should be possible to maintain both in a single repo. One example of such modification that could work in theory is the following:
repository_folder
└── pkgs
    ├── GridCal_proj
    │     ├── setup.py
    │     └── GridCal
    │         └── __init__.py
    │         └── GUI
    └── GridCalEngine_proj
        ├── setup.py
        └── GridCalEngine
            ├── __init__.py
            ├── Core
            ├── IO
            └── Simulations

So it would be nice to copy the setup (wherever it is on the structure) to the right place when building the tar.gz or whl file.

It seems here that what you are trying to achieve is "re-using" code. Is that correct?
That is not easy to achieve because the sdist created for each one of the packages still needs to be self-contained (i.e. have all the files it needs in order to be build in a fresh machine, without the other packages).
You can add a pre-processing stage that copies one file from the top of you directory to each project's root (by defining a custom build task using something like nox/tox/invoke/pydoit/...).
However the downside is that you would no longer be able to simply git clone proj && cd proj && pip install <path/inside/the/project>.
Other than that, I suppose you can also try to hook-up something with symlinks, but I haven't tested that.

I think that tools like https://www.pantsbuild.org/docs/welcome-to-pants automate this pre-processing for you. But then they use a different workflow/paradigm and you also loose the ability of running git clone proj && cd proj && pip install <path/inside/the/project>.

Just to let you know, I did something quite savage: I wrote my on packager:
"""

#!/usr/bin/env bash
python3 setup.py sdist
twine upload dist/GridCal-2.30.tar.gz

"""
import os
import sys
import tarfile
from pathlib import Path
from typing import List, Tuple
from subprocess import call
from GridCalEngine.__version__ import __GridCalEngine_VERSION__
from GridCal.__version__ import __GridCal_VERSION__

def build_setup_cfg() -> str:
    """
    Generate the content of setup.cgf
    :return:
    """
    val = '[egg_info]\n'
    val += 'tag_build = \n'
    val += 'tag_date = 0\n'

    return val

def build_pkg_info(name: str,
                   version: str,
                   summary: str,
                   home_page: str,
                   author: str,
                   email: str,
                   license_: str,
                   keywords: str,
                   classifiers_list: List[str],
                   requires_pyhon: str,
                   description_content_type: str,
                   provides_extra: str,
                   long_description: str):
    """
    Generate the content of PKG-INFO
    :param name:
    :param version:
    :param summary:
    :param home_page:
    :param author:
    :param email:
    :param license_:
    :param keywords:
    :param classifiers_list:
    :param requires_pyhon:
    :param description_content_type:
    :param provides_extra:
    :param long_description:
    :return:
    """
    val = 'Metadata-Version: 2.1\n'
    val += "Name: " + name + '\n'
    val += "Version: " + version + '\n'
    val += "Summary: " + summary + '\n'
    val += "Home-page: " + home_page + '\n'
    val += "Author: " + author + '\n'
    val += "Author-email: " + email + '\n'
    val += "License: " + license_ + '\n'
    val += "Keywords: " + keywords + '\n'

    for classifier in classifiers_list:
        val += "Classifier:  " + classifier + '\n'

    val += "Requires-Python: " + requires_pyhon + '\n'
    val += "Description-Content-Type: " + description_content_type + '\n'
    val += "Provides-Extra: " + provides_extra + '\n'

    val += '\n' + long_description + '\n'

    return val

def check_ext(filename, ext_filter) -> bool:
    """
    Check of the file complies with the list of extensions
    :param filename: filename
    :param ext_filter: list of extnsions
    :return: true/false
    """
    for ext in ext_filter:
        if filename.endswith(ext):
            return True
    return False

def find_pkg_files(path: str, ext_filter=['.py']) -> List[Tuple[str, str]]:
    """
    Get list
    :param path: path to traverse
    :param ext_filter: extensions of files to include
    :return: list of [filename, complete path
    """
    files_list = list()
    for (dirpath, dirnames, filenames) in os.walk(path):
        for fname in filenames:
            if check_ext(filename=fname, ext_filter=ext_filter):
                pth = os.path.join(dirpath, fname)
                files_list.append((fname, pth))

    return files_list

def build_tar_gz_pkg(pkg_name: str,
                     setup_path: str,
                     version: str,
                     summary: str,
                     home_page: str,
                     author: str,
                     email: str,
                     license_: str,
                     keywords: str,
                     classifiers_list: List[str],
                     requires_pyhon: str,
                     description_content_type: str,
                     provides_extra: str,
                     long_description: str,
                     folder_to_save='dist',
                     ext_filter=['py']):
    """

    :param pkg_name:
    :param setup_path:
    :param version:
    :param summary:
    :param home_page:
    :param author:
    :param email:
    :param license_:
    :param keywords:
    :param classifiers_list:
    :param requires_pyhon:
    :param description_content_type:
    :param provides_extra:
    :param long_description:
    :param folder_to_save:
    :param ext_filter:
    :return:
    """
    pkg_name2 = pkg_name + '-' + version
    filename = pkg_name2 + '.tar.gz'
    output_filename = os.path.join(folder_to_save, filename)

    files = find_pkg_files(path=pkg_name,
                           ext_filter=ext_filter)

    pkg_info = build_pkg_info(name=pkg_name,
                              version=version,
                              summary=summary,
                              home_page=home_page,
                              author=author,
                              email=email,
                              license_=license_,
                              keywords=keywords,
                              classifiers_list=classifiers_list,
                              requires_pyhon=requires_pyhon,
                              description_content_type=description_content_type,
                              provides_extra=provides_extra,
                              long_description=long_description)
    pkg_info_path = 'pkg_info' + pkg_name
    with open(pkg_info_path, 'w') as f:
        f.write(pkg_info)

    setup_cfg = build_setup_cfg()
    setup_cfg_path = 'setup_cfg' + pkg_name
    with open(setup_cfg_path, 'w') as f:
        f.write(setup_cfg)

    with tarfile.open(output_filename, "w:gz") as tar:
        for name, file_path in files:
            if not name.endswith('setup.py'):
                tar.add(file_path, arcname=os.path.join(pkg_name2, file_path))

        # add the setup where it belongs
        tar.add(setup_path, arcname=os.path.join(pkg_name2, 'setup.py'))

        # add
        tar.add(pkg_info_path, arcname=os.path.join(pkg_name2, 'PKG-INFO'))
        tar.add(setup_cfg_path, arcname=os.path.join(pkg_name2, 'setup.cfg'))

    os.remove(pkg_info_path)
    os.remove(setup_cfg_path)

    return output_filename

def read_pypirc() -> Tuple[str, str]:
    """
    Read the pypirc file located in home
    :return: user, password
    """
    home = Path.home()
    path = os.path.join(home, 'pypirc')

    with open(path) as file:
        lines = [line.rstrip() for line in file]

    user = ''
    pwd = ''
    for line in lines:
        if '=' in line:
            key, val = line.split('=')

            if key.strip() == 'username':
                user = val.strip()
            elif key.strip() == 'password':
                pwd = val.strip()

    return user, pwd

def publish(pkg_name: str,
            setup_path: str,
            version: str,
            summary: str,
            home_page: str,
            author: str,
            email: str,
            license_: str,
            keywords: str,
            classifiers_list: List[str],
            requires_pyhon: str,
            description_content_type: str,
            provides_extra: str,
            long_description: str):
    """
    Publish package to Pypi using twine
    :param pkg_name: name of the package (i.e GridCal)
    :param setup_path: path to the package setup.py (i.e. GridCal/setup.py)
    :param version: verison of the package (i.e. 5.1.0)
    :param summary:
    :param home_page:
    :param author:
    :param email:
    :param license_:
    :param keywords:
    :param classifiers_list:
    :param requires_pyhon:
    :param description_content_type:
    :param provides_extra:
    :param long_description:
    """

    # build the tar.gz file
    fpath = build_tar_gz_pkg(pkg_name=pkg_name,
                             setup_path=setup_path,
                             version=version,
                             summary=summary,
                             home_page=home_page,
                             author=author,
                             email=email,
                             license_=license_,
                             keywords=keywords,
                             classifiers_list=classifiers_list,
                             requires_pyhon=requires_pyhon,
                             description_content_type=description_content_type,
                             provides_extra=provides_extra,
                             long_description=long_description,
                             folder_to_save='dist',
                             ext_filter=['.py', '.csv', '.txt'])

    # check the tar.gz file
    call([sys.executable, '-m', 'twine', 'check', fpath])

    user, pwd = read_pypirc()

    # upload the tar.gz file
    call([sys.executable, '-m', 'twine', 'upload',
          '--repository', 'pypi',
          '--username', user,
          '--password', pwd,
          fpath])

if __name__ == "__main__":

    if __GridCalEngine_VERSION__ == __GridCal_VERSION__:  # both packages' versions must be exactly the same

        _long_description = "# GridCal \n"
        _long_description += "This software aims to be a complete platform for power systems research and simulation.)\n"
        _long_description += "\n"
        _long_description += "[Watch the video https](https://youtu.be/SY66WgLGo54)\n"
        _long_description += "[Check out the documentation](https://gridcal.readthedocs.io)\n"
        _long_description += "\n"
        _long_description += "## Installation\n"
        _long_description += "\n"
        _long_description += "pip install GridCalEngine\n"
        _long_description += "\n"
        _long_description += "For more options (including a standalone setup one), follow the\n"
        _long_description += "[installation instructions]( https://gridcal.readthedocs.io/en/latest/getting_started/install.html)\n"
        _long_description += "from the project's [documentation](https://gridcal.readthedocs.io)\n"

        _description_content_type = 'text/markdown'

        _summary = 'GridCal is a Power Systems simulation program intended for professional use and research'

        _keywords = 'power systems planning'

        _author = 'Santiago Peñate Vera et. Al.'

        _author_email =<EMAIL_ADDRESS>
        _home_page = 'https://github.com/SanPen/GridCal'

        _classifiers_list = [
            'License :: OSI Approved :: GNU Library or Lesser General Public License (LGPL)',
            'Programming Language :: Python :: 3.10',
        ]

        _requires_pyhon = '>=3.6'

        _provides_extra = 'gch5 files'

        _license_ = 'LGPL'

        publish(pkg_name='GridCalEngine',
                setup_path=os.path.join('GridCalEngine', 'setup.py'),
                version=__GridCalEngine_VERSION__,
                summary=_summary,
                home_page=_home_page,
                author=_author,
                email=_author_email,
                license_=_license_,
                keywords=_keywords,
                classifiers_list=_classifiers_list,
                requires_pyhon=_requires_pyhon,
                description_content_type=_description_content_type,
                provides_extra=_provides_extra,
                long_description=_long_description
                )

        publish(pkg_name='GridCal',
                setup_path=os.path.join('GridCal', 'setup.py'),
                version=__GridCal_VERSION__,
                summary=_summary,
                home_page=_home_page,
                author=_author,
                email=_author_email,
                license_=_license_,
                keywords=_keywords,
                classifiers_list=_classifiers_list,
                requires_pyhon=_requires_pyhon,
                description_content_type=_description_content_type,
                provides_extra=_provides_extra,
                long_description=_long_description
                )

    else:

        print(__GridCalEngine_VERSION__, 'and', __GridCal_VERSION__, "are different :(")

Thanks for the feedback @SanPen.
Yeah, building your own "packager" that does the pre-processing that you are looking after is always an option. The only thing is that it is probably incompatible with standard workflows like git clone proj && cd proj && pip install <path/inside/the/project>.
I will go ahead and close this FR as there is no plan in setuptools for adding such pre-processing capabilities.


Changing Builds service to Jobs service (v4)

This change will replace the Builds service with the Jobs service. The builds endpoints are no longer available in GitLab 9.

Thanks @bgetsug! LGTM 🎉


Add @leather.io/analytics to mobile

Updates the analytics package with more original events, segments them, adds the package to mobile in utils.
A follow up step to this PR will be to publish this package and add all the existing events in extension.

@pete-watters I advocated for that in the initial project pitch but @kyranjamie felt strongly that we should not lose historical data. I do still feel that a hard fork is better but I am not willing to rock the boat on this particular issue.


⚠ Bofrost.be has degraded performance

In 86a6e5e, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1796 ms

Resolved: Bofrost.be performance has improved in cea0fe7.


Warning fix for missing return value

Thanks! It looks like that whole function was unused, removed it altogether in https://github.com/juj/fbcp-ili9341/commit/d6fafd67bdd46876d1a8a3e53e3bd50544cc13e4.

I noticed that but wasn't sure if there was some other reason for it's continued existence :P


cxxrtl: Convert to Mem helpers.

This only does conversion, but doesn't add any new functionality —
support for memory read port init/reset is still upcoming.

TODOs (may or may not end up as separate PRs):

add initial value support for read ports
add reset support for read ports
handle memory writes in processes

Other upcoming changes:

support for meminit_v2 (memory initialization with mask)
support for per-port transparency masks (blocked on model changes)

Really like this approach!

I think it's ready for review now; some notes:

every read port is a distinct node (to avoid creating false dependencies between one port's address and another's data), but all write ports of a single memory are a single node (I wrote it this way to make it easier to handle priority, without noticing that update is already fully capable of dealing with this; we could change it if necessary)
port/memory attributes aren't printed right now, which is a minor regression; it may be a good idea to convert kernel/mem.h classes to inherit from AttrObject so that we can just dump_attrs them

Rebased with minor fix: processes with a memory write are now always inserted into live_nodes

but all write ports of a single memory are a single node (I wrote it this way to make it easier to handle priority, without noticing that update is already fully capable of dealing with this; we could change it if necessary)

This is potentially a lot better than the approach with update because sorting the queue vector takes a non-trivial amount of time in benchmarking. Since I think the concept of priority only really makes sense within a netlist (and not between co-simulating entities), we could ditch that entirely and have Memory::update do last-write-wins.

port/memory attributes aren't printed right now, which is a minor regression; it may be a good idea to convert kernel/mem.h classes to inherit from AttrObject so that we can just dump_attrs them

port/memory attributes aren't printed right now, which is a minor regression; it may be a good idea to convert kernel/mem.h classes to inherit from AttrObject so that we can just dump_attrs them

Would that be hard to do? It seems like one of those small things that otherwise easily fall through the cracks. (Not something I'd hold up the PR on, of course.)

port/memory attributes aren't printed right now, which is a minor regression; it may be a good idea to convert kernel/mem.h classes to inherit from AttrObject so that we can just dump_attrs them

Would that be hard to do? It seems like one of those small things that otherwise easily fall through the cracks. (Not something I'd hold up the PR on, of course.)

Don't think so; I'll have a shot at it tomorrow, probably.

Done, turns out it was pretty trivial.

Fixing one more bug noticed in own review: we need to take into account memory writes from processes as well when deciding whether to emit commit (ie. writable_memories is back)

The macOS builder seems stuck, trying a dummy push-force to kick it...


Flutter will provide build-time transpilation of GLSLs in the future

This is not an issue, I just want to tell you about flutter's great new feature!
Flutter provides a better way to compile GLSL into SPIR-V in the master branch:
https://github.com/flutter/flutter/issues/93498#issuecomment-1141658172

This is amazing :)


chore: bdd-spring-1580992171 to 0.0.1

chore: Promote bdd-spring-1580992171 to version 0.0.1

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


Questions About Handling NULL Values and Node Process Crash with Large Dataset

Hi,
I have two questions regarding the tool:

Is it possible to remove "NULL" values from the input? For example, in the empy-value test cases  from the KGCW challenge 2024 dataset. Is there a built-in way to handle this?

When mapping the records/1M_rows_20_columns test case from the KGCW challenge 2024 dataset my Node.js process crashes with the following error:

RangeError: Set maximum size exceeded

I ran the process with this command:
node --max-old-space-size=8000 ./src/index.js ./mapping.dot -dup_removal

I suspect this crash is caused by the duplicate removal step. Is there a way to map the data while still performing duplicate removal, or is there an alternative approach to handle this?
Thanks!

The implementation is currently on hold since the underlying algebraic formalism is still undergoing changes.
However, one thing to note is that this implementation is meant more as a platform for  fast iteration/implementation of the formalism so performance wise + edge cases is not really considered crucial here (I should put this disclaimer in the README.md).
Now onto your questions;

Currently, there is no built-in way to handle "NULL" values from the input yet.

Duplicate removal is handled at line 22 of SerializerOperator.
One way to prevent the crash, from what I've read, is to split up this duplicate bookkeeping sets into chunks as it gets "full", and
iterate through the chunks for finding duplicates as the engine processes a large dataset.

Thanks for the clarification!


Improve support for tremorsense

More specifically visions with targeting should show tokens.

Bonus: Look into custom visions.

https://github.com/foundryvtt/pf2e/blob/6c593be83336953b74c36dfc92c2e27881e58eb2/src/module/scene/token-document/document.ts#L297
https://github.com/search?q=repo%3Afoundryvtt%2Fpf2e computeSightAndDetectionForRBV&type=code


Failed to take screenshot Unknown method "_flutter.screenshot" is coming as error in Integration Testing in Chrome

When I try to take screenshots with Integration Testing then it throwing an error as -
Failed to take screenshot
_flutter.screenshot: (-32601) Unknown method "_flutter.screenshot".
Steps to Reproduce
Once app is launched in Chrome browser, try to take screenshot with BDD approach(Integration testing)
then it shows error as -
Failed to take screenshot
_flutter.screenshot: (-32601) Unknown method "_flutter.screenshot".
Expected results: Should take screenshot
Actual results: showing error in Json/html report
This is code inside app_test.dart page -
..hooks = [HookExample(), AttachScreenshotOnFailedStepHook()]

Hi @skumarpk, Thanks for filing the issue. Please share the output of flutter doctor -v, complete console logs and minimal and complete reproducible code.
Thank you.

This issue is only occurring in Chrome, same code working fine with Android device.
Please find attached flutter doctor output:

Below is the exception that occurring:
Failed to take screenshot
_flutter.screenshot: (-32601) Unknown method "_flutter.screenshot".
#0      new _OutstandingRequest (package:vm_service/src/vm_service.dart:1626:45)
#1      VmService._call (package:vm_service/src/vm_service.dart:2109:21)
#2      VmService.callServiceExtension (package:vm_service/src/vm_service.dart:2072:14)
#3      VmService.callMethod (package:vm_service/src/vm_service.dart:2062:12)
#4      VMServiceFlutterDriver.screenshot (package:flutter_driver/src/driver/vmservice_driver.dart:334:54)
#5      AttachScreenshotOnFailedStepHook.takeScreenshot (package:flutter_gherkin/src/flutter/hooks/attach_screenshot_on_failed_step_hook.dart:28:19)
#6      AttachScreenshotOnFailedStepHook.onAfterStep (package:flutter_gherkin/src/flutter/hooks/attach_screenshot_on_failed_step_hook.dart:18:32)
#7      AggregatedHook._invokeHooks (package:gherkin/src/hooks/aggregated_hook.dart:92:9)
#8      AggregatedHook.onAfterStep (package:gherkin/src/hooks/aggregated_hook.dart:85:7)
#9      FeatureFileRunner._runStep (package:gherkin/src/feature_file_runner.dart:349:5)
#10     FeatureFileRunner._runScenario (package:gherkin/src/feature_file_runner.dart:268:13)
#11     FeatureFileRunner._runScenarioInZone. (package:gherkin/src/feature_file_runner.dart:160:24)

Thanks for the info @skumarpk, flutter_gherkin  is a third-party plugin issues related to it must be filed with the maintainer. Closing as this is not an issue with flutter.


#1 - cleaning up some of the styles (and the html semantics)

I may have gotten a little carried away. I made a few css changes, and a bunch of HTML ones, and got your form looking a bit more like the original.
What I've done here:

added labels to all the inputs - very important whenever you have a form
changed the forms to type="number", since that's what they are - one bonus of this is it'll bring up a number keyboard if anyone comes to this page on their phones.
added blank alt text to the coin images (they're decorative, hence alt="" - images pretty much always need something set as the alt attribute. What that is depends on the purpose of the image.)
cleaned up the semantics; you shouldn't use <h4> (or <h1> or <h2> or any of 'em) just to get text that's large and bold. Since I wrapped the fields in fieldsets, I changed the h4s to legends.
took the content out of a <table> - that element is fine if you've got an actual table of data (you sort of do on the right side of the form), but it's not meant for page layouts. You can use css to get the same effect - here I've set the form to display: table and the two sections in it to display: table-cell, and that puts them neatly in a row next to each other.
moved all the inline styles (style="padding-right: 4px", etc.) out to the css where they're easier to manage
moved the bootstrap.css include so it comes before your custom styles; generally that's what you want to do if you're using a framework. Their styles are a base; yours come after to override them as needed. If you put your styles first and Bootstrap second, the Bootstrap styles are more likely to override what you've written.

Whew - that's a long description, but these were all small changes! I haven't touched the right side of the form since it seems like you're still working on that part, but hopefully this puts you in a good direction.

Oh wow :) looks great, didn't expect someone suddenly showing up


travis: Remove galaxy lint rules repository

The galaxy-lint-rules github repository isn't used anymore and has
been archived.
All the rules are now part of the ansible-lint project.
https://github.com/ansible/galaxy-lint-rules
https://github.com/ansible/ansible-lint
Signed-off-by: Dimitri Savineau<EMAIL_ADDRESS>

jenkins test pipeline


Add support for apps/v1 APIs

Ref https://github.com/kubernetes/kubernetes/pull/80536 and #19, seems like after https://github.com/kubernetes/kubernetes/pull/70672 workload APIs will be served solely on apps/v1. Hence autoscaler hits below error when trying to find a proper patcher (based on kind and groupVersion):
E0729 17:38:26.932770       1 autoscaler.go:49] no supported API group for target: map[apps/v1:true]

This PR adds support to recognize apps/v1 groupVersion. I will post my local test result soon.
cc @lzang @liggitt

Hitting some error:
I0730 20:28:00.196774       1 autoscaler_server.go:132] Setting nginx-vertical-autoscale-example requests["cpu"] = 500m
E0730 20:28:00.199963       1 autoscaler_server.go:144] Update failure: patch failed: the server could not find the requested resource

/hold

looks like things under https://github.com/kubernetes-incubator/cluster-proportional-vertical-autoscaler/tree/master/examples still use extensions/v1beta1 and apps/v1beta*

Okay... I ended up migrating this repo to use go module so that I can re-vendor easily. Also updated the build image to use go1.12 for compatibility reason. Example files are changed to use apps/v1 as well.
Tested on k8s cluster built from head.
/hold cancel

Lgtm, thanks

Thanks @liggitt! Adding @bowei for approval.
/assign @bowei

/approve

/lgtm


New projects licensescout-maven-utils and

New projects licensescout-maven-utils and licensescout-report-maven-plugin
utils contains utilities that are dependandt on Maven runtime system components and will be used by both maven plugin implementations that are in different projects. However, they should not be included in the core to keep the core free from Maven RT system dependencies.
licensescout-report-maven-plugin is an implementation of a Maven reporting plugin that generates a LicenseScout report for integration with the maven-site-plugin.
Type of request

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Refactoring
[ ] Documentation or documentation changes

Related Issue(s)

Concept

Description of the change

Description of current state
Description of target state
Estimated investment / approximate investment

Initiator
Matthias Pfisterer
Technical information
Platform / target area

known side-effects
other
DB Changes (if so: incl. scripts with definitions and testdata)
How has this Tested?

Screenshots, Uploads, other documents (if appropriate):

Checklist:

[x] All the commits are signed.
[x] My code follows the code style of this project.
[x] I agree with die CLA.
[x] I have read the CONTRIBUTING docs.
[x] I have added/updated necessary documentation (if appropriate).

Note: the codacy exclude is necessary because the duplicated code in AbstractScanMojo and AbstractReportMojo cannot be factored out because they are derived from different base classes of the Maven runtime system. However, for convenience of the user the maven parameters are named uniquely.


PyBuilder VEnv manager corrupts plugins venv setuptools

This could be a virtualenv or a pip bug as well.
[INFO]  Building binary distribution in /home/username/src/pybuilder/target/dist/pybuilder-0.13.0.dev
[DEBUG] Executing distutils command: ['/home/username/src/pybuilder/.pybuilder/plugins/cpython-3.9.7.final.0/bin/python', '/home/username/src/pybuilder/target/dist/pybuilder-0.13.0.dev/setup.py', '-v', 'clean', '--all', 'sdist']
[DEBUG] Executing distutils command: ['/home/username/src/pybuilder/.pybuilder/plugins/cpython-3.9.7.final.0/bin/python', '/home/username/src/pybuilder/target/dist/pybuilder-0.13.0.dev/setup.py', '-v', 'clean', '--all', 'bdist_wheel']
Traceback (most recent call last):
  File "/home/username/src/pybuilder/./src/main/python/pybuilder/cli.py", line 475, in main
    summary = reactor.build(
  File "/home/username/src/pybuilder/./src/main/python/pybuilder/reactor.py", line 235, in build
    execution_summary = self.build_execution_plan(tasks, execution_plan)
  File "/home/username/src/pybuilder/./src/main/python/pybuilder/reactor.py", line 270, in build_execution_plan
    task_execution_summaries = self.execution_manager.execute_execution_plan(
  File "/home/username/src/pybuilder/./src/main/python/pybuilder/execution.py", line 393, in execute_execution_plan
    summaries.append(self.execute_task(task, **kwargs))
  File "/home/username/src/pybuilder/./src/main/python/pybuilder/execution.py", line 337, in execute_task
    if self.execute_action(action, kwargs):
  File "/home/username/src/pybuilder/./src/main/python/pybuilder/execution.py", line 382, in execute_action
    action.execute(arguments)
  File "/home/username/src/pybuilder/./src/main/python/pybuilder/execution.py", line 95, in execute
    self.callable(*arguments)
  File "/home/username/src/pybuilder/./src/main/python/pybuilder/plugins/python/distutils_plugin.py", line 281, in build_binary_distribution
    execute_distutils(project, logger, reactor.pybuilder_venv, commands, True)
  File "/home/username/src/pybuilder/./src/main/python/pybuilder/plugins/python/distutils_plugin.py", line 394, in execute_distutils
    raise BuildFailedException(
pybuilder.errors.BuildFailedException: Error while executing setup command ['bdist_wheel']. See /home/username/src/pybuilder/target/reports/distutils/bdist_wheel for full details:
            self.run_command(cmd)
          File "/home/username/.pyenv/versions/3.9.7/lib/python3.9/distutils/dist.py", line 985, in run_command
            cmd_obj.run()
          File "/home/username/src/pybuilder/.pybuilder/plugins/cpython-3.9.7.final.0/lib/python3.9/site-packages/wheel/bdist_wheel.py", line 299, in run
            self.run_command('build')
          File "/home/username/.pyenv/versions/3.9.7/lib/python3.9/distutils/cmd.py", line 313, in run_command
            self.distribution.run_command(command)
          File "/home/username/.pyenv/versions/3.9.7/lib/python3.9/distutils/dist.py", line 985, in run_command
            cmd_obj.run()
          File "/home/username/.pyenv/versions/3.9.7/lib/python3.9/distutils/command/build.py", line 135, in run
            self.run_command(cmd_name)
          File "/home/username/.pyenv/versions/3.9.7/lib/python3.9/distutils/cmd.py", line 313, in run_command
            self.distribution.run_command(command)
          File "/home/username/.pyenv/versions/3.9.7/lib/python3.9/distutils/dist.py", line 985, in run_command
            cmd_obj.run()
          File "/home/username/src/pybuilder/.pybuilder/plugins/cpython-3.9.7.final.0/lib/python3.9/site-packages/setuptools/command/build_py.py", line 56, in run
            self.build_package_data()
          File "/home/username/src/pybuilder/.pybuilder/plugins/cpython-3.9.7.final.0/lib/python3.9/site-packages/setuptools/command/build_py.py", line 127, in build_package_data
            srcfile in self.distribution.convert_2to3_doctests):
        AttributeError: 'Distribution' object has no attribute 'convert_2to3_doctests'

Most likely virtualenv is the culprit.
$ ls -la | grep setuptools
drwxr-xr-x   7 arcivanov arcivanov   4096 Sep 16 13:25 setuptools
drwxr-xr-x   2 arcivanov arcivanov    126 Sep 16 13:25 setuptools-57.4.0.dist-info
-rw-r--r--   1 arcivanov arcivanov      0 Sep 16 13:25 setuptools-57.4.0.virtualenv
drwxr-xr-x   2 arcivanov arcivanov    143 Sep 16 13:21 setuptools-58.0.4.dist-info


How to export/import agents? or add/edit agents in CLI?

1/ There are 50+ agents at my huginn in heroku, and now they want to move to VPS. Is there any other way to move these agents of huginn from heroku to Ubuntu, except for copy and paste json of every agent one by one, which is so slow? can they be export all at once, and then all import? (only agents, no need events)
2/ Is there any other way to add/edit agents except for http? SSH/CLI is much much more faster than HTTP in both network speed and operating ways. Sometimes editing agent and troubleshooting in browser is a little annoying: slow net speed, endless buttonsss and so many tabs in browser…can agents be add/edit by CLI/SSH or manipulating some files like?
thanks.

Hi @waldens,
heroku has documentation on how to download database dumps. You could then import the database onto your VPS, the biggest problem is that we do not have a documented and tested way to run Huginn on PostgreSQL outside of heroku.
Working  on that is on my todo list and I hope to get to it on the weekend, until then a few pointers that might help:

follow the manual installation guide but replace mysql with postgresql
restore your heroku database to your local postgresql database
export ON_HEROKU=true as an environment variable before running bundle install
change the database configuration in .env to use the postgresql adapter

Sorry, kind of missed your main question. No, sadly there is no way to export all agents, if you have them grouped into scenarios you could export and import them.

I know @TildeWill was investigating this too.

Thanks @dsander ! Never noticed scenarios exist and what it is! It's exactly what I want, as you said, just group them all into scenarios and share. thanks! also thank you @cantino


Style preso creation form

This is where presenters can create a new presentation and then receive their QR code to the preso survey form.

What exactly is the desired style improvement for this page?

What exactly is the desired style improvement for this page?


Feature 368 documentation for logging

Pull Request Testing

[x] Describe testing already performed for these changes:

Verify the following:

Verify that the default config file has the log_level set to 'info' (quotes not necessary but acceptable, case can be mixed, all upper, all lower, etc)

Search the documentation for the following text (or mostly similar):

In the default config file, logging is set to stdout and the log level is INFO (i.e. any log messages
of type INFO, WARNING, and DEBUG will be logged).  If the log_filename and log_level are
not specified in the custom configuration file, these settings will be used.

and

To save the log output to a file, uncomment the log_filename entry and specify the path and
name of the log file.  Select a directory with the appropriate read and write
privileges.  To modify the verbosity of logging than what is set in the default config
file, uncomment the log_level entry and specify the log level  (debug and info are higher verbosity, warning and error are lower verbosity).

In the documentation where the custom config file is displayed, verify that there are log_filename and log_level in the config file

Visually check the Users Guide in ReadtheDocs for the changes in these plots:

bar

box

contour

eclv

ens_ss

histogram

histogram_2d

hovmoeller

line

performance diagram

reliability diagram

roc_diagram

taylor diagram

wind rose

[x] Recommend testing for the reviewer(s) to perform, including the location of input datasets, and any additional instructions:
verify the same

[x] Do these changes include sufficient documentation updates, ensuring that no errors or warnings exist in the build of the documentation? [Yes ]

[x] Do these changes include sufficient testing updates? [NA]

[x] Will this PR result in changes to the test suite? [No]
If yes, describe the new output and/or changes to the existing output:

[ ] Please complete this pull request review by [Fill in date].

Pull Request Checklist
See the METplus Workflow for details.

[ ] Review the source issue metadata (required labels, projects, and milestone).
[ ] Complete the PR definition above.
[ ] Ensure the PR title matches the feature or bugfix branch name.
[ ] Define the PR metadata, as permissions allow.
Select: Reviewer(s)
Select: Organization level software support Project or Repository level development cycle Project
Select: Milestone as the version that will include these changes
[ ] After submitting the PR, select Development with the original issue number.
[ ] After the PR is approved, merge your changes. If permissions do not allow this, request that the reviewer do the merge.
[ ] Close the linked issue and delete your feature or bugfix branch from GitHub.

@bikegeek In the right hand menu, I have linked to the issue under "Development" and have added a "Status" and "Cycle" for the Project.

@jprestop Thanks for catching the missing 'Error' in #1  I will make the addition.
for #2, I thought I committed some of the default yaml config files.
#3 and #4 Not all the plots got the logging support incorporated.  The skew-T documentation already had information about the logging, so no changes were necessary.
I'll let you know when I fix #1 and #2.

4a. equivalence_testing_bounds doesn't have content in the Users Guide, and was omitted.  It looks like this is some internal test.  I committed the custom_hovmoeller.yaml, somehow I didn't get that committed.  The difficulty_index, equivalence_testing_bounds, fv3_physics_tend, hovmoeller, mpr_plot, series_analysis, and spacetime_plot don't have logging support incorporated yet, and the skew_t test_skew_t.yaml in the test directory already has logging settings available in the config file (this was added when the skew-t diagram was created).
4b.  The subset of custom config files in the test directory are not referenced in the Users Guide and as written will use the default config values for logging.  Those tests are for development/continuous integration.


[WIP] Update fair artist view

Fixes #24.
Waiting for follow-up on https://github.com/artsy/eigen/issues/24#issuecomment-72455898.

Failure is valid – we'll need to update the screenshots in the unit tests sometime.

Would recommend running Xcode with my Snapshots plugin, makes it a one-click to replace with new.

Good idea (link).

Alcatraz is still causing the plugin to crash, so you'll need to install from source.

@ashfurrow Should be done. For a screenshot see https://github.com/artsy/eigen/issues/24#issuecomment-72666698.

Might be nice to add a or amend one of those snapshots to show what it looks like with a birthday

and BETA_CHANGELOG entry please ( it's in the xcproject )

Might be nice to add a or amend one of those snapshots to show what it looks like with a birthday

Good call. Do you want me to add a test+snapshot for the variant where either the artist’s nationality or birth year is unknown?

Also, looking at the test fixtures atm, the permutation where there is no map available has a different margin (less) than when there is a map. I assume there not being a map is a real thing to care about and thus the margin needs fixing?

Sounds right.

Yes to all of those things.
(note, needs rebase)

Done all the things.


Refactored Amcrest to use central hub component

Description: (breaking_change)
Amcrest component now has a hub to centralize its configuration which makes the Home Assistant's configuration cleaner.
This change will also help to extend the Amcrest component like the PR https://github.com/home-assistant/home-assistant/pull/7876
Pull request in home-assistant.github.io with documentation (if applicable): home-assistant/home-assistant.github.io#2881
Example entry for configuration.yaml (if applicable):
amcrest:
  - host: !secret amcrest_living
    name: "Living Room"
    username: !secret amcrest_living_username
    password: !secret amcrest_living_password
    resolution: low
    stream_source: snapshot
    sensors:
      - motion_detector
      - ptz_preset

  - host: !secret amcrest_garage
    name: "Garage"
    username: !secret amcrest_garage_username
    password: !secret amcrest_garage_password
    resolution: high
    stream_source: rtsp
    sensors:
      - sdcard
      - motion_detector

Checklist:
If user exposed functionality or configuration variables are added/changed:

[x] Documentation added/updated in home-assistant.github.io

If the code communicates with devices, web services, or third-party tools:

[x] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[x] New dependencies have been added to the REQUIREMENTS variable ([example][ex-requir]).
[x] New dependencies are only imported inside functions that use them ([example][ex-import]).
[x] New dependencies have been added to requirements_all.txt by running script/gen_requirements_all.py.
[x] New files were added to .coveragerc.

@pvizeli  thank you for your help. PR updated.
Please let me know if that needs more work.

@pvizeli  PR updated. Thanks again for your valid coaching and inputs.

@pvizeli  PR updated. Please let me know if we need to change anything else
Thanks again for your assistance.

@pvizeli  Thanks :+1:


if Step 1of1, don't show Steps

If a wizard has only one step, get rid of the label Step 1 of 1

This has already been fixed 👍


how can i use to pca9685

how can i use to pca9685

Controlling LEDs using a PCA9685 is currently not supported by JLed. I'll put a task in the backlog and do some research in the next weeks. Basically a new HAL needs to be written, which communicates with the PCA9685 using I2C.

hello i added pca9685 support to your library
I also added a 4095 gamma table for 12 bit pwm resolution.
I needed a ramp for 20 minutes or more, but I realized that the ramp that took that long does not work in your library, I couldn't fix it. Do you have a solution?

what exactly do you mean by "ramp fo 20 minutes or more"? You want a FadeOn with a period of 20 minutes?

evet 20 min and 60 min


Performance issue

@macfarla This is the test result requested by #3373.
Is this the limit of besu's performance?
I tested by setting
Transaction Gas Limit = 1,000,000,000
blockperiodseconds=2
requesttimeoutseconds=4

Block Gas Limit = 9,007,199,254,740,991 / 5 transactions per second, 400 in total.
Block Gas Limit = 9,007,199,254,740,991 / 20 transactions per second, 400 in total.
Block Gas Limit = 10,000,000,000 / 75 transactions per second, 400 in total.

Result:
1.
start: 2022-02-22 14:23:46,373 INFO o.a.j.e.StandardJMeterEngine: Running the test!
end: 2022-02-22 15:48:39.605+09:00 | EthScheduler-Workers-3 | INFO | PersistBlockTask | Imported #161 / 109 tx / 0 om / 1,807,636,604 (0.0%) gas / (0x6a127fe785bbbe8bcfe536a6eead0f9523a1bd78f8e83ecdaba1ce15496c1ac7) in 1.222s. Peers: 3

2022-02-22 14:23:50.240+09:00 | pool-8-thread-1 | INFO  | IbftBesuControllerBuilder | Imported #152 / 20 tx / 10 pending / 23,604,836 (0.0%) gas / (0x162c75e6f08643deb436c17e89adfa247af15a52259d3567ee5a8469ce037d1b)
2022-02-22 14:23:51.955+09:00 | pool-8-thread-1 | INFO  | IbftBesuControllerBuilder | Produced #153 / 10 tx / 10 pending / 19,038,292 (0.0%) gas / (0xc2014dd60d2ce30c7892058f1466171a0e65e46aabe27ce52a6caab121a38f1b)
2022-02-22 14:23:53.674+09:00 | pool-8-thread-1 | INFO  | IbftBesuControllerBuilder | Imported #154 / 10 tx / 10 pending / 18,589,264 (0.0%) gas / (0xd2e51243b3634c1a47943e8bd3706d0718d44b47e8bbe768ab98ac5a626ae212)
2022-02-22 14:23:53.675+09:00 | EthScheduler-Workers-2 | INFO  | PersistBlockTask | Imported #154 / 10 tx / 0 om / 18,589,264 (0.0%) gas / (0xd2e51243b3634c1a47943e8bd3706d0718d44b47e8bbe768ab98ac5a626ae212) in 0.013s. Peers: 3
2022-02-22 14:23:56.170+09:00 | pool-8-thread-1 | INFO  | IbftBesuControllerBuilder | Imported #155 / 10 tx / 10 pending / 23,731,353 (0.0%) gas / (0xc3b6c1236c3bc2c5615d4bc3a44eb9c985e38e8364b07872132e29045239dfd6)
2022-02-22 14:23:58.695+09:00 | pool-8-thread-1 | INFO  | IbftBesuControllerBuilder | Imported #156 / 10 tx / 10 pending / 25,909,211 (0.0%) gas / (0xc1e5da085dd7d2131fb903e863675ce0efc0e1beb0de05600071e43fef3a8d81)
2022-02-22 14:23:58.696+09:00 | EthScheduler-Workers-0 | INFO  | PersistBlockTask | Imported #156 / 10 tx / 0 om / 25,909,211 (0.0%) gas / (0xc1e5da085dd7d2131fb903e863675ce0efc0e1beb0de05600071e43fef3a8d81) in 0.001s. Peers: 3
2022-02-22 14:24:01.407+09:00 | pool-8-thread-1 | INFO  | IbftBesuControllerBuilder | Produced #157 / 10 tx / 19 pending / 27,622,494 (0.0%) gas / (0xbface2208c8759fd2362b5ef0296e02ddb2ca681782a19f8b739495cf0e1856d)
2022-02-22 14:24:07.047+09:00 | pool-8-thread-1 | INFO  | IbftBesuControllerBuilder | Imported #158 / 19 tx / 21 pending / 54,024,791 (0.0%) gas / (0x602b6e3365c0e25c9cfe4d3869e3db6f1419092a71fa1a4dc1715d1f5f7d7baa)
2022-02-22 14:24:07.048+09:00 | EthScheduler-Workers-2 | INFO  | PersistBlockTask | Imported #158 / 19 tx / 0 om / 54,024,791 (0.0%) gas / (0x602b6e3365c0e25c9cfe4d3869e3db6f1419092a71fa1a4dc1715d1f5f7d7baa) in 0.012s. Peers: 3
2022-02-22
14:24:12.748+09:00 | EthScheduler-Workers-1 | INFO  | PersistBlockTask | Imported #159 / 21 tx / 0 om / 51,551,605 (0.0%) gas / (0xc55968ca81501a980e21b2e4da78f594630be900b547e579e83412196bc53cb3) in 0.023s. Peers: 3
2022-02-22 14:24:12.748+09:00 | pool-8-thread-1 | INFO  | IbftBesuControllerBuilder | Imported #159 / 21 tx / 30 pending / 51,551,605 (0.0%) gas / (0xc55968ca81501a980e21b2e4da78f594630be900b547e579e83412196bc53cb3)
2022-02-22 14:54:02.410+09:00 | pool-8-thread-1 | INFO  | IbftRound | Importing block to chain. round=ConsensusRoundIdentifier{Sequence=160, Round=8}, hash=0x2a9f52dbc15c669c295a0dd2eeb1119a0548d89ef72821a6a2d9b0a811b88216
2022-02-22 14:55:58.616+09:00 | pool-8-thread-1 | INFO  | IbftBesuControllerBuilder | Imported #160 / 181 tx / 109 pending / 1,527,252,909 (0.0%) gas / (0x2a9f52dbc15c669c295a0dd2eeb1119a0548d89ef72821a6a2d9b0a811b88216)
2022-02-22 15:44:51.084+09:00 | pool-8-thread-1 | INFO  | IbftRound | Importing block to chain. round=ConsensusRoundIdentifier{Sequence=161, Round=9}, hash=0x6a127fe785bbbe8bcfe536a6eead0f9523a1bd78f8e83ecdaba1ce15496c1ac7
2022-02-22 15:48:39.605+09:00 | pool-8-thread-1 | INFO  | IbftBesuControllerBuilder | Imported #161 / 109 tx / 0 pending / 1,807,636,604 (0.0%) gas / (0x6a127fe785bbbe8bcfe536a6eead0f9523a1bd78f8e83ecdaba1ce15496c1ac7)
2022-02-22 15:48:39.605+09:00 | EthScheduler-Workers-3 | INFO  | PersistBlockTask | Imported #161 / 109 tx / 0 om / 1,807,636,604 (0.0%) gas / (0x6a127fe785bbbe8bcfe536a6eead0f9523a1bd78f8e83ecdaba1ce15496c1ac7) in 1.222s. Peers: 3`

start: 2022-02-22 16:13:26,675 INFO o.a.j.e.StandardJMeterEngine: Running the test!
end: 2022-02-23 02:03:07.732+09:00 | pool-8-thread-1 | INFO  |

2022-02-22 16:13:24.032+09:00 | pool-8-thread-1 | INFO  | IbftBesuControllerBuilder | Imported #560 / 0 tx / 0 pending / 0 (0.0%) gas / (0x0fc22bb2eb04597574ba4c3a8eefb7da7d6423e2a1dcb21d8515f610bec8d2d8)
2022-02-23 01:15:35.233+09:00 | pool-8-thread-1 | INFO  | IbftRound | Importing block to chain. round=ConsensusRoundIdentifier{Sequence=561, Round=12}, hash=0xdf883c1a0ff2e4fe711bcdcb75e87260684681f3a7f859738195a0e19c1e04e5
2022-02-23 02:03:07.732+09:00 | pool-8-thread-1 | INFO  | IbftBesuControllerBuilder | Produced #561 / 400 tx / 0 pending / 12,843,261,226 (0.0%) gas / (0xdf883c1a0ff2e4fe711bcdcb75e87260684681f3a7f859738195a0e19c1e04e5)

start: 2022-02-23 11:18:55,928 INFO o.a.j.e.StandardJMeterEngine: Running the test!
end: 2022-02-23 13:07:32.271+09:00 | pool-8-thread-1 | INFO  |

2022-02-23 11:18:56.019+09:00 | pool-8-thread-1 | INFO  | IbftBesuControllerBuilder | Imported #52 / 0 tx / 0 pending / 0 (0.0%) gas / (0x40fefcbec1f4c1e325340c6eb1826a6d4e5a36739b9bbe51d243610b6291821e)
2022-02-23 12:59:33.825+09:00 | pool-8-thread-1 | INFO  | IbftRound | Importing block to chain. round=ConsensusRoundIdentifier{Sequence=53, Round=10}, hash=0xa3c817268308cd9d9e8a0ac0f3c725774d7b26bdb4c5cd853fac847824467693
2022-02-23 13:07:32.271+09:00 | pool-8-thread-1 | INFO  | IbftBesuControllerBuilder | Imported #53 / 400 tx / 0 pending / 4,308,298,326 (43.1%) gas / (0xa3c817268308cd9d9e8a0ac0f3c725774d7b26bdb4c5cd853fac847824467693)

Hi @SeongUkJeon, that's awesome, you found some interesting things with your testing. With IBFT2 and blockchain in general, a lot depends on your individual environment - network latency, hardware, etc.
It looks like you've made some headway to finding the optimal tuning settings for your environment.
From here, you could try

increasing the blockperiodseconds - 2s is still quite short.
decreasing the block gas limit - the higher this value, the bigger the blocks will be and the longer it will take to produce and sign them.
It depends on your use case too - if bigger blocks is going to make your life easier, then you will need to increase the blockperiodseconds. If you are aiming for more smaller, more frequent blocks, then you will need to decrease the block size (block gas limit).

@macfarla
Thank you for your answer.
As we continue to test, we will find the appropriate optimal value.
If there is a problem, I will contact here again.
Thanks once again.

@macfarla
Hi,
I found the right value.
It is blockperiodseconds 30 and block gasLimit 10,000,000,000.
I have a question here.
What is the relationship between blockperiodseconds and block size?
If blockperiodseconds are low, even a few transactions should be written to the block and mined, but why can't you create a block at all?

Glad you found the right value for you environment!
There's no direct relationship between blockperiodseconds and block size - it's more that if the block takes longer to produce, there will be more transactions in it. There's a limit on block size which is measured in gas.
If blockperiodseconds is too low, there isn't enough time for the validators to exchange the messages required to agree on the block. This depends mostly on network latency - how long does it take for the validators to exchange messages.


exit status 255 or hang on Google CE

When running following command...
docker-machine create --driver google --google-project app-inst-ae --google-zone europe-west1-c --google-username<EMAIL_ADDRESS>staging-test4

On version 0.1.0 I get the following error
INFO[0009] Saving token in /Users/robwithhair/.docker/machine/machines/staging-test2/gce_token
INFO[0009] Creating host...
INFO[0010] Generating SSH Key
INFO[0010] Creating instance.
INFO[0011] Waiting for Instance...
INFO[0027] Waiting for SSH...
INFO[0075] Uploading SSH Key
INFO[0076] Waiting for SSH Key
INFO[0079] Configuring Machine...
ERRO[0079] Error creating machine: exit status 255
WARN[0079] You will want to check the provider to make sure the machine and associated resources were properly removed.
FATA[0079] Error creating machine

On version v0.2.0-rc1 it just hangs waiting for SSH key like the following
INFO[0009] Saving token in /Users/robwithhair/.docker/machine/machines/staging-test4/gce_token
INFO[0009] Creating host...
INFO[0010] Generating SSH Key
INFO[0011] Creating instance.
INFO[0012] Waiting for Instance...
INFO[0029] Waiting for SSH...
INFO[0076] Uploading SSH Key
INFO[0076] Waiting for SSH Key

When looking on GCE it looks like the key has been created as expected though.  I'm not sure where to go from here so any info or help of how to debug further would be much appreciated.

Doesn't look like it can communicate with the instance for some reason

OK, all of these issues was because I set --google-username<EMAIL_ADDRESS>when it should have been --google-username rob, if required at all.  It isn't the username you log in with to google it is the username for the machine.  Better documentation perhaps on this particular option.  I thought this was required too.
Not closing as documentation could use improvement but not a bug.

@robwithhair thanks for reporting and digging through the issue!  we'll get the Google docs updated for the proper usage.

Thanks!  Machine really is great.

@robwithhair thanks!

If i want to pass the boot disk as something else instead of Default (ubuntu-1404-trusty-v20150128) as well then?

@thebaron93 there needs to be a flag added (https://github.com/docker/machine/issues/1243)

Closing.  Please open a new issue if you continue to have trouble.  Thanks!


feat: Complex rotation

WIP

it looks a lot like weighted sum. If there's anything to be done in complex space, well this isn't it.


[SMP-2190] SMP 0.9.3 release notes

SMP 0.9.3 release notes
For reviewers, a preview is available.
https://6530296269983b4830e581ec--harness-developer.netlify.app/release-notes/self-managed-enterprise-edition#093
What Type of PR is This?

[ ] Issue
[ ] Feature
[x] Maintenance/Chore

If tied to an Issue, list the Issue(s) here:

Issue(s)

House Keeping
Some items to keep track of. Screen shots of changes are optional but would help the maintainers review quicker.

[x] Tested Locally
[ ] Optional Screen Shoot.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app

@brian-f-harness This seems to be missing https://harness.atlassian.net/browse/CHAOS-2606?

I have added a commit updating the same https://github.com/harness/developer-hub/pull/3786#issuecomment-1774445749

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app


Splattershot MPU Wrong Value

The MPU values for Splattershot are incorrect. The curve should be the same as .52 Gal.

Working on a large update that will be updating all weapon damage values and MPU parameters.

 Update ALL Weapon Damage and MPU Param Updates (using 4.7 data)

07415a4927ce0a5f0df168eafccbde56af7fe6f8 Fixes this. However, it won't be pushed to the live beta until the MPU calculations are updated.

The live beta now has all of the MPU changes. Please confirm you see the fix and close the issue if it works, or let me know if you're still seeing incorrect values.


[WIP] quarantine tests which were reported as failing in testgrid

What this PR does / why we need it:
Do NOT merge.
/hold
This PR quarantines tests which occured in test grid as failing. The goal is to find passing subset of tests.
/cc @dankenigsberg
Special notes for your reviewer:
Release note:
NONE

Signed-off-by: Karel Šimon<EMAIL_ADDRESS>

/retest
to see if this was only luck

it passed for 3 times in row, lets see if it fail tomorrow.

closing in favour of https://github.com/kubevirt/kubevirt/pull/5284


Fix: STFC Tech Reviewers not loading

Closes https://github.com/UserOfficeProject/issue-tracker/issues/1012
Description
This PR fixes the issue of STFC Tech Reviewers not loading in the proposal view.
Motivation and Context
The change was required to solve the problem where the Tech Reviewers for STFC were not being loaded
Changes

Added the getProposalsFromView method in StfcProposalDataSource to include details of technical reviewers in the proposal views fetching them from the UOWS.

Tests included/Docs Updated?

[ ] I have added tests to cover my changes.
[ ] All relevant doc has been updated

Sorry it took me so long to look at this.

No worries!


DB::Exception: inflate failed: buffer error

Hi,  when inserting to Clickhouse with HTTP transport and gzip (compression level 1), I see a lot of stack traces like the following. They don't happen regularly (sometimes the pipeline keeps running without errors for a long time), I see these errors usually when the size of the gzipped data reaches around 100MB+.
There are three different stack traces which are common:
2017.01.27 00:59:40.444106 [ 10 ] <Error> executeQuery: Code: 354, e.displayText() = DB::Exception: inflate failed: buffer error: (at row 47486), e.what() = DB::Exception (from [::ffff:<IP_ADDRESS>]:52380) (in query: INSERT INTO r0.some_table_replica FORMAT TabSeparated  ), Stack trace:

0. /usr/bin/clickhouse-server(StackTrace::StackTrace()+0x16) [0x122ad76]
1. /usr/bin/clickhouse-server(DB::Exception::Exception(std::string const&, int)+0x1f) [0xf8792f]
2. /usr/bin/clickhouse-server(DB::ZlibInflatingReadBuffer::nextImpl()+0x277) [0x1240537]
3. /usr/bin/clickhouse-server(DB::ConcatReadBuffer::nextImpl()+0x3e) [0xf8fc5e]
4. /usr/bin/clickhouse-server(DB::ConcatReadBuffer::nextImpl()+0x3e) [0xf8fc5e]
5. /usr/bin/clickhouse-server(DB::TabSeparatedRowInputStream::read(DB::Block&)+0x31c) [0x25f80cc]
6. /usr/bin/clickhouse-server(DB::BlockInputStreamFromRowInputStream::readImpl()+0x3d) [0x261449d]
7. /usr/bin/clickhouse-server(DB::IProfilingBlockInputStream::read()+0x2a0) [0x10d3060]
8. /usr/bin/clickhouse-server(DB::InputStreamFromASTInsertQuery::readImpl()+0x18) [0x119dfb8]
9. /usr/bin/clickhouse-server(DB::IProfilingBlockInputStream::read()+0x2a0) [0x10d3060]
10. /usr/bin/clickhouse-server(DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::atomic<bool>*)+0x5c) [0x1d300fc]
11. /usr/bin/clickhouse-server(DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, DB::Context&, std::shared_ptr<DB::IBlockInputStream>&, std::function<void (std::string const&)>)+0x285) [0x119d8e5]
12. /usr/bin/clickhouse-server(DB::HTTPHandler::processQuery(Poco::Net::HTTPServerRequest&, HTMLForm&, Poco::Net::HTTPServerResponse&, DB::HTTPHandler::Output&)+0x10de) [0xf8e10e]
13. /usr/bin/clickhouse-server(DB::HTTPHandler::handleRequest(Poco::Net::HTTPServerRequest&, Poco::Net::HTTPServerResponse&)+0x222) [0xf8f812]
14. /usr/bin/clickhouse-server(Poco::Net::HTTPServerConnection::run()+0x27b) [0x305e74b]
15. /usr/bin/clickhouse-server(Poco::Net::TCPServerConnection::start()+0xf) [0x3042e5f]
16. /usr/bin/clickhouse-server(Poco::Net::TCPServerDispatcher::run()+0x10b) [0x30609eb]
17. /usr/bin/clickhouse-server(Poco::PooledThread::run()+0x87) [0x30d8707]
18. /usr/bin/clickhouse-server(Poco::ThreadImpl::runnableEntry(void*)+0x96) [0x309c086]
19. /lib/x86_64-linux-gnu/libpthread.so.0(+0x80a4) [0x7f7b51eaa0a4]
20. /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f7b514d262d]

2017.01.27 19:15:59.631346 [ 70 ] <Error> HTTPHandler: Code: 354, e.displayText() = DB::Exception: inflate failed: buffer error, e.what() = DB::Exception, Stack trace:

0. /usr/bin/clickhouse-server(StackTrace::StackTrace()+0x16) [0x122ad76]
1. /usr/bin/clickhouse-server(DB::Exception::Exception(std::string const&, int)+0x1f) [0xf8792f]
2. /usr/bin/clickhouse-server(DB::ZlibInflatingReadBuffer::nextImpl()+0x277) [0x1240537]
3. /usr/bin/clickhouse-server(DB::ConcatReadBuffer::nextImpl()+0x3e) [0xf8fc5e]
4. /usr/bin/clickhouse-server(DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, DB::Context&, std::shared_ptr<DB::IBlockInputStream>&, std::function<void (std::string const&)>)+0x6fc) [0x119dd5c]
5. /usr/bin/clickhouse-server(DB::HTTPHandler::processQuery(Poco::Net::HTTPServerRequest&, HTMLForm&, Poco::Net::HTTPServerResponse&, DB::HTTPHandler::Output&)+0x10de) [0xf8e10e]
6. /usr/bin/clickhouse-server(DB::HTTPHandler::handleRequest(Poco::Net::HTTPServerRequest&, Poco::Net::HTTPServerResponse&)+0x222) [0xf8f812]
7. /usr/bin/clickhouse-server(Poco::Net::HTTPServerConnection::run()+0x27b) [0x305e74b]
8. /usr/bin/clickhouse-server(Poco::Net::TCPServerConnection::start()+0xf) [0x3042e5f]
9. /usr/bin/clickhouse-server(Poco::Net::TCPServerDispatcher::run()+0x10b) [0x30609eb]
10. /usr/bin/clickhouse-server(Poco::PooledThread::run()+0x87) [0x30d8707]
11. /usr/bin/clickhouse-server(Poco::ThreadImpl::runnableEntry(void*)+0x96) [0x309c086]
12. /lib/x86_64-linux-gnu/libpthread.so.0(+0x80a4) [0x7fac06e5d0a4]
13. /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fac0648562d]

2017.01.27 13:32:10.967702 [ 29 ] <Error> HTTPHandler: Code: 354, e.displayText() = DB::Exception: inflate failed: data error, e.what() = DB::Exception, Stack trace:

0. /usr/bin/clickhouse-server(StackTrace::StackTrace()+0x16) [0x122ad76]
1. /usr/bin/clickhouse-server(DB::Exception::Exception(std::string const&, int)+0x1f) [0xf8792f]
2. /usr/bin/clickhouse-server(DB::ZlibInflatingReadBuffer::nextImpl()+0x277) [0x1240537]
3. /usr/bin/clickhouse-server(DB::ConcatReadBuffer::nextImpl()+0x3e) [0xf8fc5e]
4. /usr/bin/clickhouse-server(DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, DB::Context&, std::shared_ptr<DB::IBlockInputStream>&, std::function<void (std::string const&)>)+0x6fc) [0x119dd5c]
5. /usr/bin/clickhouse-server(DB::HTTPHandler::processQuery(Poco::Net::HTTPServerRequest&, HTMLForm&, Poco::Net::HTTPServerResponse&, DB::HTTPHandler::Output&)+0x10de) [0xf8e10e]
6. /usr/bin/clickhouse-server(DB::HTTPHandler::handleRequest(Poco::Net::HTTPServerRequest&, Poco::Net::HTTPServerResponse&)+0x222) [0xf8f812]
7. /usr/bin/clickhouse-server(Poco::Net::HTTPServerConnection::run()+0x27b) [0x305e74b]
8. /usr/bin/clickhouse-server(Poco::Net::TCPServerConnection::start()+0xf) [0x3042e5f]
9. /usr/bin/clickhouse-server(Poco::Net::TCPServerDispatcher::run()+0x10b) [0x30609eb]
10. /usr/bin/clickhouse-server(Poco::PooledThread::run()+0x87) [0x30d8707]
11. /usr/bin/clickhouse-server(Poco::ThreadImpl::runnableEntry(void*)+0x96) [0x309c086]
12. /lib/x86_64-linux-gnu/libpthread.so.0(+0x80a4) [0x7fac06e5d0a4]
13. /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fac0648562d]

Hi. Thank you for the report!
Does it reproduce if you change compression level to 2?
There could be several reasons:

We recently have changed usual zlib library to zlib-ng. It could be related. Try to get reproducible test case: could you create one file, which always triggers the error?
Then we will try to debug. Even if file will contain confidential data, we could debug by trying different variants of library on your side, without disclosure of data.

Broken network connection. In this case, you should look at error message, that you get on client side.

And also quick question. If we will add support of Brotli (Content-Encoding: br), it will be more efficient than gzip both in terms of compression ratio and compression speed. Do you appreciate?

Hi, a couple more questions: how do you create your compressed data? Is it with stock zlib? Can you make sure that the stream is indeed correct (i.e. stock gzip decompresses it without problems)? There are bugs in level 1 compression algorithm in some of the zlib forks floating around (including zlib-ng and Intel zlib) so it makes sense to first ensure that the stream is correct.

We use cgzip, not sure what gzip lib is it linked against but it's possible it's broken only with the best speed compression level. It's working fine with v1.1.54127-stable

It's working fine with v1.1.54127-stable

That means the compressed data is OK and the problem is either with zlib-ng or our code. I'm going to create a branch with zlib-ng substituted for stock zlib after the weekend. It would be great if you'll be able to build it and test against it.

Does it reproduce if you change compression level to 2?

I deployed inserters with compression level 2, got the same error (v1.1.54134-stable)
2017.01.29 22:44:36.505169 [ 352 ] <Error> executeQuery: Code: 354, e.displayText() = DB::Exception: inflate failed: buffer error: (at row 221474), e.what() = DB::Exception (from [::ffff:<IP_ADDRESS>]:51294) (in query: INSERT INTO r0.requests_error_replica FORMAT TabSeparated  ), Stack trace:

0. /usr/bin/clickhouse-server(StackTrace::StackTrace()+0x16) [0x122ad76]
1. /usr/bin/clickhouse-server(DB::Exception::Exception(std::string const&, int)+0x1f) [0xf8792f]
2. /usr/bin/clickhouse-server(DB::ZlibInflatingReadBuffer::nextImpl()+0x277) [0x1240537]
3. /usr/bin/clickhouse-server(DB::ConcatReadBuffer::nextImpl()+0x3e) [0xf8fc5e]
4. /usr/bin/clickhouse-server(DB::ConcatReadBuffer::nextImpl()+0x3e) [0xf8fc5e]
5. /usr/bin/clickhouse-server(DB::TabSeparatedRowInputStream::read(DB::Block&)+0x31c) [0x25f80cc]
6. /usr/bin/clickhouse-server(DB::BlockInputStreamFromRowInputStream::readImpl()+0x3d) [0x261449d]
7. /usr/bin/clickhouse-server(DB::IProfilingBlockInputStream::read()+0x2a0) [0x10d3060]
8. /usr/bin/clickhouse-server(DB::InputStreamFromASTInsertQuery::readImpl()+0x18) [0x119dfb8]
9. /usr/bin/clickhouse-server(DB::IProfilingBlockInputStream::read()+0x2a0) [0x10d3060]
10. /usr/bin/clickhouse-server(DB::copyData(DB::IBlockInputStream&, DB::IBlockOutputStream&, std::atomic<bool>*)+0x5c) [0x1d300fc]
11. /usr/bin/clickhouse-server(DB::executeQuery(DB::ReadBuffer&, DB::WriteBuffer&, DB::Context&, std::shared_ptr<DB::IBlockInputStream>&, std::function<void (std::string const&)>)+0x285) [0x119d8e5]
12. /usr/bin/clickhouse-server(DB::HTTPHandler::processQuery(Poco::Net::HTTPServerRequest&, HTMLForm&, Poco::Net::HTTPServerResponse&, DB::HTTPHandler::Output&)+0x10de) [0xf8e10e]
13. /usr/bin/clickhouse-server(DB::HTTPHandler::handleRequest(Poco::Net::HTTPServerRequest&, Poco::Net::HTTPServerResponse&)+0x222) [0xf8f812]
14. /usr/bin/clickhouse-server(Poco::Net::HTTPServerConnection::run()+0x27b) [0x305e74b]
15. /usr/bin/clickhouse-server(Poco::Net::TCPServerConnection::start()+0xf) [0x3042e5f]
16. /usr/bin/clickhouse-server(Poco::Net::TCPServerDispatcher::run()+0x10b) [0x30609eb]
17. /usr/bin/clickhouse-server(Poco::PooledThread::run()+0x87) [0x30d8707]
18. /usr/bin/clickhouse-server(Poco::ThreadImpl::runnableEntry(void*)+0x96) [0x309c086]
19. /lib/x86_64-linux-gnu/libpthread.so.0(+0x80a4) [0x7f05271720a4]
20. /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f052679a62d]

Could you try to reproduce with the version compiled from this branch?
Meanwhile I'll try to reproduce the issue on our servers.

I will try this out very soon.

Also in current master now possible to build with -DUSE_INTERNAL_ZLIB_LIBRARY=0
it will use your system zlib

So, I had been putting off reproducing this for sometime (as we started running our inserters without compression and it wasn't hurting.)
However, during the weekend, while debugging some other issue I realized that this bug was NOT a problem with ClickHouse or zlib. It was a bug in our inserter. When our inserter is unable to insert to one Clickhouse node it tries to insert to another node and so on. This works fine when the insertion failure happens before we send any data. Sometimes, however, the insertion failure happens after we have sent some data because of broken streams etc. In that case, we were not resetting our buffers and sending a bad stream which was causing the  "inflate failed: buffer error" at the server.
The broken streams were only happening when the stream size was big, that's why we were not seeing it regularly and only seeing it when the gzipped stream size was 100MB+. That is also the reason why we were not seeing it with the v1.1.54127-stable version, we have two ClickHouse clusters running, the one running v1.1.54127-stable never reached 100MB+ gzipped streams while inserting.
Thanks a lot for looking into this issue guys. Really sorry for wasting your time on this. :(

No problem and thank you for investigating.